Identifying robots (was Re: Open Source Proxy)

[Petite Abeille]

> On Jul 23, 2020, at 22:45, Sean Conner <sean at conman.org> wrote:
> 
> How do I instruct a Gemini bot with robots.txt,
> when there's no way for a Gemini bot to identify itself?

Through a side channel such as the TLS certificate? Robots could identify themselves there.

Otherwise, only User-agent: * wide rules apply.

Alternatively, some creative use of TLS fingerprinting.