Identifying robots (was Re: Open Source Proxy)

Sean Conner sean at conman.org
Thu Jul 23 21:45:50 BST 2020


It was thus said that the Great Jason McBrayer once stated:
> 
> This is cool, but when you stand it up, don't forget an appropriate
> robots.txt! 

  Question---HTTP has the Use-Agent: header to help identify webbots, but
Gemini doesn't have that.  How do I instruct a Gemini bot with robots.txt,
when there's no way for a Gemini bot to identify itself?

  -spc



More information about the Gemini mailing list