RFC 9039 (robots.txt) has been published. Note that 11 % of capsules known by the Lupa crawler have a robots.txt (each with a slightly different syntax). https://www.rfc-editor.org/info/rfc9039
2 years ago
https://www.rfc-editor.org/info/rfc9039
gemini://gemini.bortzmeyer.org/rfc-mirror/rfc9309.txt · 2 years ago
gemini://gemini.bortzmeyer.org/rfc-mirror/rfc9309.txt
Too bad it is apparently not possible to edit but, yes, it is 9309, sorry. · 2 years ago
Wrong number?
https://www.rfc-editor.org/rfc/rfc9309.html · 2 years ago
https://www.rfc-editor.org/rfc/rfc9309.html
That links to a document titled Uniform Resource Names for Device Identifiers. How is that related to robots.txt? · 2 years ago