💾 Archived View for bbs.geminispace.org › u › skyjake › 19366 captured on 2024-12-17 at 15:36:00. Gemini links have been rewritten to link to archived content
-=-=-=-=-=-=-
Re: "Search Engines in Gemini"
@istvan:
Didn’t think linking to them was a requirement.
It likely isn't required... @Acidus will be able to elaborate if this is just a matter of the crawler getting around to indexing the capsule.
Sep 03 · 3 months ago
I think it may have been due to my robots.txt, which was there in an attempt to stop the content from being hit by web crawlers if someone linked to a document over a Gemini to Web proxy.
My attempt to block spiders from caching may have blocked Kennedy from indexing.
Search Engines in Gemini — I found out that when I search for my capsule on any of the search engines on Gemini, nothing comes up? I tried adding its url in Kennedy as well, few times, still nothing. Its weird since, the crawlers should be working just fine since I do not even have robot.txt in my main path of the capsule. Did anyone else has experienced this before? Its been more than 10 days since I launched my capsule. I hope it shows up in the search at least, just in case someone is cuious...
💬 Aptor-theHobbit · 7 comments · Sep 02 · 3 months ago · #capsule #search_engines