💾 Archived View for rawtext.club › ~sloum › geminilist › 005414.gmi captured on 2024-02-05 at 11:11:08. Gemini links have been rewritten to link to archived content

-=-=-=-=-=-=-

[SPEC] Encouraging HTTP Proxies to support Gemini hosts self-blacklisting

Johann Galle johann at qwertqwefsday.eu

Sun Feb 21 20:48:13 GMT 2021

- - - - - - - - - - - - - - - - - - -

Hi,

why is robots.txt not the obvious answer here? The companion specification[1] has a "User-agent: webproxy" for this specific case:

### Web proxies

Gemini bots which fetch content in order to translate said content into HTML and publicly serve the result over HTTP(S) (in order to make Geminispace accessible from within a standard web browser) should respect robots.txt directives aimed at a User-agent of "webproxy".

So this should suffice:


Regards,Johann

-- You can verify the digital signature on this email with the public keyavailable through web key discovery. Try e.g. `gpg --locate-keys`...or go to<https://qwertqwefsday.eu/.well-known/openpgpkey/hu/spd3xecxhotzgyu1p3eqdqdp31ba6rif>.

-------------- next part --------------A non-text attachment was scrubbed...Name: OpenPGP_0xA4EFCC5A6174FB0F.ascType: application/pgp-keysSize: 3131 bytesDesc: not availableURL: <https://lists.orbitalfox.eu/archives/gemini/attachments/20210221/1221b622/attachment-0001.bin>-------------- next part --------------A non-text attachment was scrubbed...Name: OpenPGP_signatureType: application/pgp-signatureSize: 840 bytesDesc: OpenPGP digital signatureURL: <https://lists.orbitalfox.eu/archives/gemini/attachments/20210221/1221b622/attachment-0001.sig>