💾 Archived View for jb55.com › ward.asia.wiki.org › federation-scraper captured on 2021-12-05 at 23:47:19. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2021-12-04)
-=-=-=-=-=-=-
Here we supervise the ongoing scrape of the wiki federation. We invision this as cooperating loops where sitemap fetches lead to page fetches and these lead to more sitemap fetches.
See Stepping the Async Scrape
While developing this technology we focus first on a nested loop. We have several versions of this where we explore different instrumentation strategies.
This page has been forked from a federation outpost that wraps a substantial, long-running, and data intense application with a wiki friendly interfaces for operating and documenting work done there.
See Federation Friendly Outposts
The implementation takes a fresh look at all implementation decisions. This is not our first step back, but it is the rethink that has us most excited today.
The usual federation navigation mechanisms may not be available yet. But more than you might expect works. Try the mock computation, a triple loop running in a San Francisco datacenter. lineup github