Sean Conner sean at conman.org
Thu Oct 21 02:19:49 BST 2021
- - - - - - - - - - - - - - - - - - -
It was thus said that the Great Anna “CyberTailor” once stated:
On 2021-10-20 19:39, Sean Conner wrote:
Oh, and one more thing---my Gemini site has been down for the past few
days. Why? Mainly because I got tired of poorly written Gemini bots stuck
in my redirection tests for *weeks!* If the authors don't care that their
bots are wasting time and resources on following endless redirects, then I
don't care about outing these idiots. These are the top 10 bots over the
past four weeks:
This is disappointing. Gemini was meant for humans but got full of
stupid* bots (like everything on the net).
* disrespecting robots.txt, the protocol spec and common sense
Yeah, tell me about it. Here's the robots.txt I had (have):
User-agent: *Disallow: /test/redirhell
User-agent: archiverDisallow: /test/redirhellDisallow: /boston
And the majority of traffic to my site? Bots. Crawling through/test/redirhell or /boston.
Fail2ban or "44 slow down" can be used to stop them though.
Yes. Or you know, just as easy to stop running the server.
-spc