Sotiris Papatheodorou sotirisp at protonmail.com
Thu Sep 3 10:17:06 BST 2020
- - - - - - - - - - - - - - - - - - -
On Wednesday, September 2, 2020, Brian Evans wrote:
I think if something like `robots.txt` were to be used for this
purpose I would recommend doing it at the directory level (and thus
break from how robots.txt works).
On Wednesday, September 2, 2020, Brian Evans wrote:
To that, I would also put in a vote for
an opt-in system rather than an opt-out system (like robots.txt).
Agreed on both points! I was thinking of implementing a personal archiving system like the one mentioned by Solderpunk ingopher://zaibatsu.circumlunar.space:70/0/~solderpunk/phlog/the-individual-archivist-and-ghosts-of-gophers-past.txt