💾 Archived View for thrig.me › blog › 2024 › 05 › 19 › gemfeed2atom.gmi captured on 2024-06-19 at 22:49:06. Gemini links have been rewritten to link to archived content

-=-=-=-=-=-=-

gemfeed2atom

Previously shell and Perl were dabbled with for minimal conversion of gemfeed to Atom, with various assumptions and simplifications. This version of the same minimalism is in C:

gemfeed2atom.c

The gemtext is assumed to have a title line. It would be more efficient to parse this from a -t flag, but then you might end up double-entering the title in multiple places, and thus end up with two or more titles if you randomly change one and forget to update the other.
The links are assumed to be relative to the base URL. If not, you'll need to pull in URI parsing code.
The quoted XML strings are assumed to only be used in particular places in the XML (text nodes or in attributes) so the code cannot flexibly account for those places changing. An XML library would help, but the goal is to minimize dependencies (a C99 compiler on a unix-like system (which is a lot of bloat)).
Links are assumed to be subscription links, and as such "=> blah blah de blah blah" would fail as the YYYY-MM-DD check will fail. My blog index page has no such links, but others may want to ignore such non-matching links instead of an error being thrown.
Any other problems not mentioned here.

Efficiency

If judged by "time spent putting the code together" the C is very inefficient. On the CPU (and memory usage) front, we might compare the C with a library-using Perl script.

gem2atom.pl

    $ fgrep -c '=> ' index.gmi
    360
    $ time ./gemfeed2atom -b x index.gmi >/dev/null
        0m00.01s real     0m00.01s user     0m00.00s system
    $ time ./gem2atom.pl -b x index.gmi >/dev/null
        0m03.86s real     0m02.97s user     0m00.81s system

That was with the CPU set as slow as possible, which is how I usually have it set, except when playing silly games that need the CPU or doing something with audio that needs more CPU. With the CPU turned all the way up (this makes the fans run), the runtime is less bad for Perl:

    $ time ./gemfeed2atom -b x index.gmi >/dev/null
        0m00.00s real     0m00.00s user     0m00.00s system
    $ time ./gem2atom.pl -b x index.gmi >/dev/null
        0m00.55s real     0m00.45s user     0m00.08s system

But the speed difference is significant enough to not need a longer benchmark; languages or implementations that are closer in speed will need longer and more careful checks to suss out whether there really is a noteworthy difference.

Some of the slowness here is FFS on OpenBSD, especially when large numbers of files are involved, as will be common in Perl scripts or other such languages that pull in various modules. Hence me porting a lot of my scripts to C; the spinny metal harddrive on the 2009 MacBook (RIP 2022) also demanded such speed improvements. The various modules may also be prone to security or supply chain attacks, and could rack up quite a lot of code that several someones would ideally need to review for any such problems (but probably will not). On the other hand, the C is pretty fragile, not very flexible, and may not generate correct XML, so there are tradeoffs here.

P.S. I also have a Go version of the script, but that pulls in random Atom and RSS modules, which fail to put an ultimate '\n' at the end of the file, and most of my Go scripts eventually end up being rewritten in C.

P.P.S. I'm dogfooding the C code here, so if the Atom is broke, that's probably why.