💾 Archived View for rawtext.club › ~sloum › geminilist › 005492.gmi captured on 2021-11-30 at 19:37:34. Gemini links have been rewritten to link to archived content
-=-=-=-=-=-=-
Mansfield mansfield at ondollo.com
Tue Feb 23 01:41:43 GMT 2021
- - - - - - - - - - - - - - - - - - -
On Sun, Feb 21, 2021 at 6:44 PM Sean Conner <sean at conman.org> wrote:
It was thus said that the Great Mansfield once stated:
I must admit, I'm woefully lacking skill or background with robots.txt.
It
seems like it could be a great answer.
A few questions to help me educate myself:
<snip
More about it can be read here [2].
-spc
[1]
https://portal.mozz.us/gemini/gemini.circumlunar.space/docs/companion/robots.gmi
[2] http://www.robotstxt.org/robotstxt.html
Thanks for the links.
I'll add this to the TODO list.-------------- next part --------------An HTML attachment was scrubbed...URL: <https://lists.orbitalfox.eu/archives/gemini/attachments/20210222/c593548a/attachment.htm>