Logging format for Gemini servers

[Stephane Bortzmeyer]

On Mon, Jul 19, 2021 at 03:22:49AM +0500,
 Anna “CyberTailor” <cyber@sysrq.in> wrote 
 a message of 91 lines which said:

> Almost every HTTP server uses NCSA Common Log Format (or its
> superset - Combined Log Format). This is very cool, because
> developers of misc utilities (like fail2ban or monitoring tools)
> don't need to bother writing log parsers for each server.

Yes, this is cool but it doesn't mean this format is perfect. The
biggest problem is that it logs the source IP address but not the
source port. Because of the importance of IP address sharing today in
the IPv4 world (RFC 6269
<gemini://gemini.bortzmeyer.org/rfc-mirror/rfc6269.txt>), logging just
the source IP address is a bad idea (RFC 6302
<gemini://gemini.bortzmeyer.org/rfc-mirror/rfc6302.txt> recommends,
*if* you log the source IP address, to log also the source port).

Also, of course, there is the privacy issue. IMHO, Gemini servers
should offer an option to log only the first N bits of the source IP
address.

>  |         | |     .---- datetime string [%d/%b/%Y:%H:%M:%S %z]

RFC 3339 <gemini://gemini.bortzmeyer.org/rfc-mirror/rfc3339.txt>
format would have been a better idea.

> Thankfully, Gemini doesn't require client identification as there're no
> compatibility issues between different Gemini clients. But that makes
> learning anything about robots very hard for capsule operators :(

Indeed, this is a serious operational problem. There have been some
attempts to list all "good" robots somewhere but it was not a success.

> I appreciate Stéphane Bortzmeyer for including additional info in
> robots.txt requests:
> 
> > gemini://example.space/robots.txt?robot=true&uri=gemini://gemini.bortzmeyer.org/software/lupa/

Note that it breaks some Gemini servers
<https://framagit.org/bortzmeyer/lupa/-/issues/9>.

> Downsides:
> * identd probably won't work behind ISP's NAT
> * requires writing asynchronous or threaded server code to avoid
>   blocking main thread (although separating logger and listener
>   processes is a good idea as it's more secure)

Indeed. It seems to me there are serious limitations.