💾 Archived View for bbs.geminispace.org › s › Cosmos › 665 captured on 2023-06-16 at 17:29:16. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2023-06-14)
-=-=-=-=-=-=-
Currently, there's the ability to limit displayed posts to recent ones, which is great.
On this page, there's a dynamic link to only display newer ones than the ones displayed on the recent posts page:
Unfortunately, this link is crawled by crawlers and ends up on search engine result pages. This is not ideal, because if I search for an uncommon keyword, where I can estimate the number of expected results, e.g. my last name, then I still get ~800 results, and the vast majority of them are links to Cosmos' "Show newer posts than this":
Geminispace.info search for nordmeyer
Blocking `/~Cosmos/since` in robots.txt seems like a good idea.
Any thoughts on this?
2023-05-21 · 4 weeks ago
Agreed, I'll add that to robots.txt. That should alleviate the issue at least.