💾 Archived View for rawtext.club › ~sloum › geminilist › 002317.gmi captured on 2020-09-24 at 03:12:15. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2020-09-24)
-=-=-=-=-=-=-
Sean Conner sean at conman.org
Thu Jul 23 21:45:50 BST 2020
- - - - - - - - - - - - - - - - - - -
It was thus said that the Great Jason McBrayer once stated:
This is cool, but when you stand it up, don't forget an appropriate
robots.txt!
Question---HTTP has the Use-Agent: header to help identify webbots, butGemini doesn't have that. How do I instruct a Gemini bot with robots.txt,when there's no way for a Gemini bot to identify itself?
-spc