2021-12-11

Towards a proper FlightLog 10 -- A Library to cure Link Rot?

#flightlog

A little while ago, I wrote about link rot in gemini space.

Apparently link-rot is larger than I expected. So maybe you want to routinely mirror some content as well.

/en/2021/20211128-link-rot.gmi

I even mused about mirroring some content locally.

Example

alex@nytpu.com had a reply to my post mentioned above.

gemini://nytpu.com/gemlog/2021-11-28.gmi

A downloaded copy of their post is stored locally in the library:

/library/nytpu.com/gemlog/2021-11-28.gmi

Please note, that any of the in-capsule links within that copy will not work here. But the text at least should be there. This link shall serve as a proof of concept.

Options

I could, of course, download copies of other capsules' content and use a tool like agunua/geminitrack or similar. However, I'm afraid that doing so will put enough weight on my capsule as to disturb it's orbit. :-)

https://framagit.org/bortzmeyer/agunua

I played with gcat and a shell script to prepare the directory tree and call it on the desired URL. Needless to say, I could "of course" make this shell script smart and make it automatically download any links it finds, when scanning my new post, retreive and store the content of those links locally and even add a reference into the text. Beware: Laziness in perfection ahead, which might well last a life time.

https://github.com/aaronjanse/gcat

https://notabug.org/tinyrabbit/gemcall

I can M-x elpher-download the page I'm looking at and create some directory structure manually like top_dir / library / name_of_site / path_to_file and put the file there. Somewhat tedious, but possible. Could probably be enhanced by a few clever lines of emacs lisp.

https://thelambdalab.xyz/git/elpher.git

I could use some other client with built-in caching (I did not check). However, I would prefer the cached content in file form. This way I can easily include them into my capsule for others to read.

Details

/library/index.gmi

I found like 80 links to external gemini documents in my capsule, so there's some work ahead.

Along the Way

At the very least this whole experiment proved, what I knew before: I have no bloody clue about networking protocols. No wonder, since I have been largely following the "everything is a filter" abstraction put forward in "The Unix Programming Environment" by Brian Kernighan and Rob Pike.

My hope is to provide an idea, of how to prevent link rot, while not accumulating all of gemini space into one place.

However, one could dream BIG as well: volunteers set up a number of servers, which provide "The Gemini Archive" service, similar to the Internet Archive. Geminauts then register their capsules at one or more of them, similar to registering their feed to Antenna. Everytime they ping the archive server, that server will retrieve the whole capsule and store it. Some cleverness to avoid storage bloat would sure help (git offers help :). So a few fairly complete "The Gemini Archive" sites are built over time, offering public content.

Cheers,

~ew

Home