💾 Archived View for sysrq.in › en › gemlog › nitter.gmi captured on 2023-07-10 at 13:32:16. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2023-01-29)
-=-=-=-=-=-=-
Elon Musk: *destroys Twitter*
Me, who put a lot of effort to packaging Nitter:
So, this is my first gemlog item on the English variant of my capsule. I'd like to tell you how I packaged Nitter for Gentoo.
Nitter is written in Nim (hence the name):
It was already packaged by xgqt so I could skip this step.
By the way, I really liked that language - it's very appealing and expressive!
The problem was their package manager/build system, called Nimble. It didn't work in network-sandboxed environments, didn't have staged installation support and there was no way to tell if build/test stage could be run without invoking these commands.
GNU Coding Standards: Support for Staged Installs
Implementing --offline flag was relatively easy, and it was accepted upstream:
Nimble PR #967: Add --offline option
I tried to add "dry run" support but implementing it neatly was not possible due to Nimble's architecture, so it was rejected:
Nimble PR #964: Add --dryRun option
Support for staged installs was done easily but upstream wasn't really fond of this idea, as they didn't understand that filesystem access is sandboxed in package managers. So I had to implement more complicated thing (that never went upstream).
Nimble PR #965: nimble install: add --destDir option
Nimble PR #969: Support multiple Nimble dirs
I did some research on how other distributions package Nim software. Arch, as always, did it the way upstream intended - using nimble. Debian specified all dependencies on the command line manually to avoid using nimble. Nix had its own helper package.
nim_builder: Custom Nim builder for Nixpkgs
I decided to write my own build system, as that was also a great opportunity to learn Ninja syntax. And then procrastinated a lot (a really lot). But suddenly I was hit with hypomania and started coding. I can't recreate the history exactly because I used "git rebase -i" many times. However, the main steps were:
That was easy, as there's a Python module for generating ninja.build files in Ninja sources, and Nim syntax closely resembles Python.
Nimble has its own format for package metadata. It lacks formal specification, but kind-of documented in ReadMe. There's also a deprecated format (which Nimble still uses internally) but I didn't want support for it, so I went for a newer NimScript-based one.
To make quuerying work, I had to copy the nimble file to a temporary directory and change its file extension to "nims". Weird bu ok.
I copy-pasted this bit from Nimble and removed code that was not needed. Same with parsing command-line arguments and populating the list of files to install.
Finding packages by name is easy: just scan build root for matching ${pkgname}-${pkgver} directories.
But there are dependencies that are specified only by its URL. So I decided to put the canonical package URL to a nimblemeta.json file to use it later.
My build system creates build targets for every binary and task specified in the nimble file. Output messages will certainly remind you of CMake :)
Also it generates scripts to be called by "ninja install" and "ninja test". They are written in NimScript to be portable.
Initally, every value was read by executing the nimble file. It was slow and inefficient.
I thought that multithreading could help but soon discovered that the number of bugs can be calculated using the formula "x = a^n", where n is the number of threads.
So I redesigned it again to read all values in batch.
I didn't like how binaries were being rebuilt every time, even when sources hadn't been changed. The solution mentioned in Ninja docs is makefile-syntax dependency files.
I prepared a "pull request" for Nim to add support for it, but upstream got angry at me and closed it. No wonders there's a hard fork of Nim floating around…
Nim PR #19960: Add depfile option
That led me into maintaing my patch as a separate project, see:
So here am I, maintaining 30 Nim packages in ::guru ebuild repository. And Nitter (the main consumer) has very unclear fate since the Twitter deal.
My build system is called nimbus, although internally it's called NimBS.
You can ask me anything about it on #nimbus (at irc.oftc.net) or #nimbus:matrix.org.
I hope it will find use outside of Gentoo packaging.
Thanks for reading it!
--
