robots.txt for Gemini formalised

[Drew DeVault]

A web portal is a one-to-one mapping of a user request to a gemini
request. It's not an automated process. It's a genuine user agent, an
agent of a user. The level of traffic you'd receive from a web portal is
similar to the amount of traffic you'd receive from any other user
agent, and rate controls or access blocking don't make sense.

As the maintainer of such a web portal, I officially NACK any suggestion
that it should obey robots.txt, and will not introduce such a feature.