News----good, bad, ugly? You decide (was Re: [spec] comments on the proposed gemini spec revisions)

[Sean Conner]

It was thus said that the Great Anna “CyberTailor” once stated:
> On 2021-10-20 19:39, Sean Conner wrote:
> >   Oh, and one more thing---my Gemini site has been down for the past few
> > days.  Why?  Mainly because I got tired of poorly written Gemini bots stuck
> > in my redirection tests for *weeks!*  If the authors don't care that their
> > bots are wasting time and resources on following endless redirects, then I
> > don't care about outing these idiots.  These are the top 10 bots over the
> > past four weeks:
> 
> This is disappointing. Gemini was meant for humans but got full of
> stupid* bots (like everything on the net).
> 
> * disrespecting robots.txt, the protocol spec and common sense

  Yeah, tell me about it.  Here's the robots.txt I had (have):

User-agent: *
Disallow: /test/redirhell

# Following content a mirror of http://boston.conman.org/

User-agent: archiver
Disallow: /test/redirhell
Disallow: /boston

  And the majority of traffic to my site?  Bots.  Crawling through
/test/redirhell or /boston.  

> Fail2ban or "44 slow down" can be used to stop them though.

  Yes.  Or you know, just as easy to stop running the server.

  -spc