[tech] robots.txt format

[Stephane Bortzmeyer]

On Wed, Jan 27, 2021 at 10:40:39AM +0100,
 René Wagner <rwagner at rw-net.de> wrote 
 a message of 23 lines which said:

> simple question:

Complicated answers:

> is the following robots.txt format valid in a form that the
> "disallow" is applied to all User-agents mentioned before?

1) There is no standard for robots.txt.

2) There is not yet an "official" adaptation to Gemini, just
proposals.