Today was one of those days. Load climbing up to 40 because of some idiot writing a bot following every single link on Emacs Wiki. Don’t do that! Be mindful all the pitfalls out there:
1. robots.txt for disallowed URLs, crawl delay, and more
2. meta tags for robots in the HTML you’re getting in order to decide whether to follow any links or not
3. using common sense
4. keeping an eye on the logs
Don’t be an idiot.
#Web #Administration #Butlerian Jihad
