Crawlers on Gemini and best practices

Côme Chilliet come at chilliet.eu
Thu Dec 10 14:07:50 GMT 2020
I don't see anything in the spec saying to stop at first match. I think you should read the whole response and apply all lines that matches your virtual user agent.
So in this case for an archiver, all lines.


Archive home | About this archive