💾 Archived View for rawtext.club › ~sloum › geminilist › 002342.gmi captured on 2020-09-24 at 03:11:09. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2020-09-24)
-=-=-=-=-=-=-
Solderpunk solderpunk at posteo.net
Sat Jul 25 16:04:40 BST 2020
- - - - - - - - - - - - - - - - - - -
On Fri Jul 24, 2020 at 12:01 AM CEST, Natalie Pendragon wrote:
There's been some talk of the generic sorts of user-agents in the
past, which I think is a really nice idea. If `indexer` is a
user-agent that both sites and crawlers had some sort of informal
consensus on, then sites wouldn't need to worry about keeping up with
any new indexers popping up.
Some other generic user-agent ideas, iirc, were `archiver` and
`proxy`.
I still really like this idea. It will be a long and tediousundertaking to build up some kind of rough consensus on a set ofuser-agents with good coverage and granularity of different scenarios,but I think it might be worth the effort.
It would be great to finally get a proper "robots.txt for Gemini" sidespec written up.
Cheers,Solderpunk