💾 Archived View for jb55.com › ward.asia.wiki.org › start-or-stop-the-scrape captured on 2021-12-05 at 23:47:19. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2021-12-04)
-=-=-=-=-=-=-
An inital scrape can take the better part of a day. Press 'start' to begin. Shift-'start' to do one cycle at a time.
We start scrapes hourly skipping hours if they run long.
We fetch sitemaps for one site and then discover more.
We fetch page json to index and inspect for more sites.
See Queue Stats while running.
See also Active Sites, Failed Sites, Crazy Business