File size issues

[Jason McBrayer]

solderpunk writes:

> This also, of course, sets a precendent of "whenever we decide a little
> bit of extra metadata would be handy in the header, just append it after
> a tab", which over time could bloat our header until it's basically just
> a HTTP header in disguise with tabs instead of newlines.

We are, of course, re-inventing the Content-Length header here, which is
an extremely useful header so that clients can do things like have a
progress bar for downloads. If we support every useful feature, of
course, we end up with basically HTTP 0.9.

> Sean writes:
>>   One way would be to query a well-known endpoint (these exist in the HTTP
>> world---robots.txt is one such file) that contains tiemstamps for various
>> resources.  Slap a MIME type of text/gemini-timestamp and call it done:
>>
>> gemini://example.com/	2019-08-15T13:53:00-05:00
>> gemini://example.com/feed	2019-07-29T00:00:00-05:00
>> gemini://example.com/other	2019-08-01T00:00:00-05:00
>>
>>   That's one way.

IMO you probably don't want to serve the timestamps or file sizes for
every single file/resource in one response, for a number of reasons both
privacy-related and performance-related.

> I actually quite like this idea.  No need to make it timestamp-specific
> either.  We could have a well-known endpoint for general file metadata,
> which listed modification time, file size, checksum, MIME type, etc.  It
> could accept queries for a specific path, and *that* could be the way to
> do an equivalent of a HEAD request.

Yeah, it makes sense to bundle the metadata into one response, but only
serve for one path per request.

> This would let clients for specific scenarios do the extra work
> themselves to work around their problems, e.g. clients with very low
> memory or storage space could request the metadata for all files before
> attempting to access them and warn the user if the file size exceeds a
> threshold; clients on unreliable connections could request the metadata
> before downloading and then warn the user if file size and/checksum did
> not agree.  Most "normal" clients could do neither and just operate as
> they already do.

The one downside I see to that is that once we start seeing more complex
clients written, especially graphical clients, we will start seeing
progress bars, download managers, etc., and there will always be a need
to make a second request to get the metadata, which is relatively
expensive.

> I think it's kind of neat to keep solutions to edge problems outside of
> the protocol itself and push them into things like well-known endpoints
> like the above where they can easily be ignored when they are not
> needed/wanted.  The downside is that server developers have to do the
> work to add support for these things - but it's expected, I think, that
> servers are harder to write than clients.

My experience *so far* is that server implementation is easier. But that
is likely to change, I guess.

>>   As I mentioned in a private email to solderpunk earlier, one could always
>> take advantage of the sub-delimeters in the path portion.  I had at one
>> point mentioned using those to specify the prompt (otherwise the server
>> would return a status of 10):
>>
>> 	gemini://example.com/search;Search%20for
>>
>> This could be formalized:
>>
>> 	gemini://example.com/search;prompt=Search%20for
>> 	gemini://example.com/blogfeed;timestamp=2019-08-15T00:00:00Z
>> 	gemini://example.com/wildexample;prompt=Search%20for;timestamp=2019-08-15:00:00:00Z?query=foo&usename=bar
>>
>>   So, you have "prompt" and "timestamp".  Others could be propsed.  If the
>> "timestamp" thing above is accepted, then you might want to have a new
>> status code meaning "no change" or "okay, but there's no content".

This is definitely re-inventing HTTP headers with a different format.

--
+----------------------------------------------------------------------+
| Jason F. McBrayer                                jmcbray at carcosa.net |
| The scalloped tatters of the King in Yellow must hide Yhtill forever.|