💾 Archived View for geminispace.org › s › Cosmos › 665 captured on 2023-05-24 at 18:06:19. Gemini links have been rewritten to link to archived content
-=-=-=-=-=-=-
Currently, there's the ability to limit displayed posts to recent ones, which is great.
On this page, there's a dynamic link to only display newer ones than the ones displayed on the recent posts page:
Unfortunately, this link is crawled by crawlers and ends up on search engine result pages. This is not ideal, because if I search for an uncommon keyword, where I can estimate the number of expected results, e.g. my last name, then I still get ~800 results, and the vast majority of them are links to Cosmos' "Show newer posts than this":
Geminispace.info search for nordmeyer
Blocking `/~Cosmos/since` in robots.txt seems like a good idea.
Any thoughts on this?
2023-05-21 · 3 days ago
Agreed, I'll add that to robots.txt. That should alleviate the issue at least.