Web proxies and robots.txt (was Re: Heads up about a Gemini client @ 198.12.83.123)

[Sean Conner]

It was thus said that the Great Sean Conner once stated:
> 
> you would probably be amazed at just how often clients *don't* limit
> following redirects.  Most of the time, someone is sitting there, watching
> their client and stopping it after perhaps 30 seconds, fix the first
> redirect issue (redirecting back to itself) only to get trapped at the next
> step.

  I want to clarify that this usually happens when a new client is being
tested.  Somehow, the web proxy in question wasn't tested.

  -spc