💾 Archived View for rawtext.club › ~sloum › geminilist › 005492.gmi captured on 2023-11-04 at 13:56:20. Gemini links have been rewritten to link to archived content

-=-=-=-=-=-=-

[SPEC] Encouraging HTTP Proxies to support Gemini hosts self-blacklisting

Mansfield mansfield at ondollo.com

Tue Feb 23 01:41:43 GMT 2021

- - - - - - - - - - - - - - - - - - -

On Sun, Feb 21, 2021 at 6:44 PM Sean Conner <sean at conman.org> wrote:

It was thus said that the Great Mansfield once stated:

I must admit, I'm woefully lacking skill or background with robots.txt.

It

seems like it could be a great answer.

A few questions to help me educate myself:

<snip

More about it can be read here [2].

-spc

[1]

https://portal.mozz.us/gemini/gemini.circumlunar.space/docs/companion/robots.gmi

[2] http://www.robotstxt.org/robotstxt.html

Thanks for the links.

I'll add this to the TODO list.-------------- next part --------------An HTML attachment was scrubbed...URL: <https://lists.orbitalfox.eu/archives/gemini/attachments/20210222/c593548a/attachment.htm>