External SHA hashes instead? (was: Re: Content length, EOF -- ways to resolve whether we received everything)

[Solderpunk]

On Sat Oct 31, 2020 at 10:21 AM CET, Björn Wärmedal wrote:
> It wouldn't just be about content integrity; it's also about caching. If
> the client has fetched a file before, and it hasn't changed since,
> there's
> no need to waste time and bandwidth fetching it again.

I'm not philosophically opposed to caching or anything - far from it -
but it comes with a complexity cost and whether or not that cost is
worth paying depends upon the benefits it brings.

On the web, caching is a win, because webpages are composed of many
different parts (style sheets, background images, tracking scripts,
sometimes even content!), some of those parts are re-used across
multiple pages, and some of those parts can be fairly large.

In Geminispace, the majority of content is small (a kilobyte or so),
and visiting a single URL only means downloading an individual resource,
so there's no way for a resource to be frequently re-used like a large
background image might be on a website.  I'm not sure the cost to
benefit ratio is quite so clearly favourable.

It's true that frequent use of "the back button" or similar
navigational strategies to get around Geminispace means that certain
individual pages may be loaded several times within a browsing session
(one might go back and forth between a page listing entries in a gemlog
and those individual entries), and that can be a little wasteful.
Smarter navigation tools (like AV-98's "tours") can reduce this waste,
but it's also easily fixed by clients doing something simple like
keeping the 5 or 10 most recently visited pages cached in memory, with
cached pages expiring after 10 minutes or so (AV-98 does this, although
I don't think I've done a release since adding it).  Clients doing this
would obviously have problems with content which updated more frequently
than every 10 minutes, but that kind of content is - and I would argue
*should* be - very rare in Geminispace.  Apps are the one obvious place
where this strategy might fail, which is perhaps yet another argument in
favour of having different clients optimised for reading and for apps.

Cheers,
Solderpunk