💾 Archived View for silentcrescent.org › tech › software › language-package-managers.gmi captured on 2021-12-04 at 18:04:22. Gemini links have been rewritten to link to archived content

-=-=-=-=-=-=-

Language Package Managers

Many languages today have their own package managers. Some notable examples include:

Cabal for Haskell

Stack for Haskell

Cargo for Rust

Opam for OCaml

... and many others. They are extremely convenient tools to developers, as they allow you to specify all the dependencies needed for your software project, often automatically install and compile everything correctly and manage it over time. Compared to ecosystems that are based on Makefiles, autotools, CMake and other similar tools, they make life much easier for developers.

Annoyances

These tools, however, do not come without their annoyances. None of these are fundamental to language package managers and many can easily be addressed with some work and a change fo mindset. That being said, they have not been addressed and at the time of this writing, these problems remain one of the main detractors for adoption (at least when building systems software and considering deployment to many users).

Forceful static linking. The topic of static vs dynamic linking isn't really relevant here. The main point is that a tool such as a language package manager should not make an assumption of what the user needs. You need to support both options to be useful. As a user, I don't want the full system built in a language whose main package manager forces static linking and makes me download 200 gigabytes of data because libstring has a 1 line fix;

Lack of external dependency management (not all have this problem). A language package manager that can't specify dependencies on system packages, such as libfoo>=1.0 is not a very useful one for real world software;

Trying to control the entire ecosystem. This one is a bit odd. Normally, the main benefit of a language package manager is that developers get full control of their software. They simply need to specify the dependencies in a file and the package manager handles it, right? Well, not really. System package managers exist for a good reason: to make a well-integrated, sane system that users can consume. Unfortunately, with language package managers that try to veto what the system thinks should happen, there is no updating the software as the distribution/OS sees fit. As a developer, you are effectively stating: "It's my way or the highway.". This might be fine if you are 100% inversted into your project. Unfortunately, most developers move on from their current pet project and stop maintaining it, either due to changing jobs, lack of time, etc. Normally, people might fork the project or system package managers might simply update the libraries as they see fit and run the tests. Not in this case. If the language package manager vetos the decision of a system package manager to link against a newer version of a library and fails to compile, the system maintainers have a real problem. They might need to ship a vulnerable system, maintain a local fork specific to their distro, or simply remove the package. Gone are the days where the distribution could simply continue to ship your software and update the libraries for you, when you don't have time in order to have their system well-integrated and patched.

When we combine all of these points together, we end up with an unmaintainable mess with outdated software that is just waiting to be exploited and end up in dependency hell. I like modern languages that help developers write their software more easily and in a safer way. However, this attitude of language designers having opinions on how software should be developed needs to go away. It has no basis in reality. You simply can't predict how someone might want to use your language, so make your deployment story flexible.

In case you wish to dive deeper into the story of packaging software written in such a language, take a look at what GNU Guix, a packaging system that aims to be both bootstrappable and reproducible in its builds and acts as a 'git repository for your system' does for Rust crates:

GNU Guix's crates-io.scm

GNU Guix's crates-graphics.scm

GNU Guix's crates-gtk.scm

Note how many duplicate entries of the same library exist. Now note that a lot of these libraries don't need to stay on the same version. In fact, a lot of them could just be squashed into one single library, as they are backwards compatible. cargo prevents this from happening because it vetos the decision on which library version is acceptable. The only options are roughly as described above:

Patch the Cargo.toml file locally;
Duplicate everything (potentially ship vulnerable versions);
Don't package it.

A lot of these languages are great in many ways, but this is holding them back. The deployment story of these languages is anything but sensible, and it needs to be fixed.

-- tdg

Silent Crescent Home