💾 Archived View for ew.srht.site › en › 2021 › 20211211-flightlog-10.gmi captured on 2022-03-01 at 15:22:24. Gemini links have been rewritten to link to archived content

-=-=-=-=-=-=-

2021-12-11

Towards a proper FlightLog 10 -- A Library to cure Link Rot?

tags: flightlog

A little while ago, I wrote about link rot in gemini space.

Apparently link-rot is larger than I expected. So maybe you want to routinely mirror some content as well.

/en/2021/20211128-link-rot.gmi

I even mused about mirroring some content locally.

Example

alex@nytpu.com had a reply to my post mentioned above.

gemini://nytpu.com/gemlog/2021-11-28.gmi

A downloaded copy of their post is stored locally in the library:

/library/nytpu.com/gemlog/2021-11-28.gmi

Please note, that any of the in-capsule links within that copy will not work here. But the text at least should be there. This link shall serve as a proof of concept.

Options

I could, of course, download copies of other capsules' content and use a tool like agunua/geminitrack or similar. However, I'm afraid that doing so will put enough weight on my capsule as to disturb it's orbit. :-)

https://framagit.org/bortzmeyer/agunua

I played with gcat and a shell script to prepare the directory tree and call it on the desired URL. Needless to say, I could "of course" make this shell script smart and make it automatically download any links it finds, when scanning my new post, retreive and store the content of those links locally and even add a reference into the text. Beware: Laziness in perfection ahead, which might well last a life time.

https://github.com/aaronjanse/gcat

https://notabug.org/tinyrabbit/gemcall

I can M-x elpher-download the page I'm looking at and create some directory structure manually like top_dir / library / name_of_site / path_to_file and put the file there. Somewhat tedious, but possible. Could probably be enhanced by a few clever lines of emacs lisp.

https://thelambdalab.xyz/git/elpher.git

I could use some other client with built-in caching (I did not check). However, I would prefer the cached content in file form. This way I can easily include them into my capsule for others to read.

Details

There is an additional directory "library" in my capsule now.
Please find a direct reference from the front page of may capsule.
Every time I refer to someone elses text, I shall download a copy of said text into the library.
And in addition to the original reference I will add a link to the copy in the library.
The Library shall have their own, commented index.
I will refrain from "fixing" references within copied files in the library.
A date of download might be desirable, the original link and a comment that the authors are not me.

/library/index.gmi

I found like 80 links to external gemini documents in my capsule, so there's some work ahead.

Along the Way

the generation of the capsules indices are too much dependant on my capsules' specifics. Some adaption was needed.
one has to deal with % escapes in URLs

At the very least this whole experiment proved, what I knew before: I have no bloody clue about networking protocols. No wonder, since I have been largely following the "everything is a filter" abstraction put forward in "The Unix Programming Environment" by Brian Kernighan and Rob Pike.

My hope is to provide an idea, of how to prevent link rot, while not accumulating all of gemini space into one place.

However, one could dream BIG as well: volunteers set up a number of servers, which provide "The Gemini Archive" service, similar to the Internet Archive. Geminauts then register their capsules at one or more of them, similar to registering their feed to Antenna. Everytime they ping the archive server, that server will retrieve the whole capsule and store it. Some cleverness to avoid storage bloat would sure help (git offers help :). So a few fairly complete "The Gemini Archive" sites are built over time, offering public content.

Cheers,

~ew

Home