💾 Archived View for michal_atlas.srht.site › posts › primitives.gmi captured on 2024-08-24 at 23:35:01. Gemini links have been rewritten to link to archived content

-=-=-=-=-=-=-

You might be surprised[a] to see both Guix and Nix in the tags today.

That’s because I decided to look into something that’s been sitting in the back of my mind for a few years now.

There’s these two awesome projects, that sadly don’t really want to talk nice with each other. So what if, we made something that could be the middleman.

Today we’ll just be looking at the ideas and possibilities, nothing concrete.

The ideal outcome is that we write something, and then we can run both guix build and nix build on it, in both cases receiving a successful output.

In a perfect world we would be able to simply translate, one to the other. And while I’d have more faith in being able to, disassemble Guix expressions into their more fundamental parts and trying to mangle them in a way that Nix might have any idea what they mean, it’s still basically impossible, as both are changing ecosystems with different capabilities. Who’s to say that one of those expressions wouldn’t be doing something Nix simply can’t do, as it’s not a general purpose language.

So that’s probably not the way to go. We could create a separate system, that would transpile to these two languages. But how would we do that? Since, as I mentioned they are a constantly changing target. We must target the lowest-level commonality, between the two, the derivation.

What are Derivations?

A derivation is the basic building block of a functional package manager. One could argue that they are what actually makes one be one.

A derivation essentially has inputs, outputs and a builder.

Inputs are paths in the store that contain, already built software and files.
Outputs are paths where you’re expected to actually put the result of the derivation.
The builder is the thing that gets run (as far as I know, it’s an exec call on the file) to turn the inputs into an output.

Essentially a derivation is the information that you want to do: lisp (apply #'builder inputs) => output

And the daemon makes sure that only the directories listed as inputs are accessible. No other directories, no internet, no nothing. The builder is completely sandboxed while it’s doing its job.

There are .drv files that describe these derivations to the daemon that are still identical in both Guix and Nix thanks to them both relying on these same primitives. The .drv files are written in ATerm[a] which seems like an interesting point of Ingress, however it is way too low-level to leverage any of the tooling and Guix explicitly marks this as an implementation detail subject to change.

[a]

Both systems use these as their building blocks:

builtins.derivation {
    name = "...";
    builder = "...";
    args = [ ... ];
    ...
}

(derivation store "name" builder '(args ...))

And in actuality, actually everything, can be built this way, since the interesting stuff, like populating /etc, creating symlinks, manipulating profiles, starting services, are done through activation scripts. So anything we’d like to do, is expressed by, building a derivation that results in a script, which we run to set the rest up.

This doesn’t seem like that hard of a target, to be able to convert to.

There’s one extra hurdle though, that is fixed-output derivations.

Since sometimes you need an input that isn’t already on your system, i.e. downloading the source code of the program in the first place. There’s the second type that isn’t sandboxed, but must explicitly state what file with what hash will be produced. I’m not entirely sure how these work yet, so I’ll have to look into them.

Plugging into the Ecosystems

I wasn’t that nervous about Guix, since you can just import a library, and you’re off to the races, evaluating and transpiling to your hearts content.

Nix has Flakes, and to be usable, we need those to work with whatever approach we come up with. Which is luckily quite straightforward as well.

;;; FILE: packages.lisp
(require 'uiop)
(uiop:with-output-file (f (uiop:getenv "out"))
  (format f "pkgs: { packages.x86_64-linux = { ~{~a = ~a;~} }; }"
          (mapcan
           (lambda (q)
             (list
              (string (aref q 0))
              (concatenate 'string "pkgs." q)))
           `("sbcl" "ecl" "abcl"))))

The script is just a simple proof of concept, that prints a Nix expression to the default output of a derivation, usually passed as the $out environment variable.

A file then contains a simple derivation, that evaluates the script with sbcl, in a derivation:

;;; FILE: lispflake.nix
pkgs:
(import (builtins.derivation {
  name = "lispflake";
  builder = "${pkgs.sbcl}/bin/sbcl";
  args = [ "--script" "${./packages.lisp}" ];
  system = "x86_64-linux";
})) pkgs

So the output of the derivation is the file containing a Nix expression.

Afterwards we issue an import (which takes a Nix file, and returns the expression it contains, similar to Scheme’s load), on the file that is generated by the derivation.

It also needs pkgs, so that it can get sbcl from them, and pass it to the attrset created by the previous script.

Finally we plug this into a flake.

;;; FILE: flake.nix
{
  inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
  outputs = { nixpkgs, ... }:
    let pkgs = import nixpkgs { system = "x86_64-linux"; };
    in import ./lispflake.nix pkgs;
}

There’s an issue here with the fact that this derivation, must be built during the time Nix only expects to evaluate. Nix separates evaluation of Nix and realization of builds, as much as it possibly can. So that for example, their indexing engine or build queue doesn’t build too much stuff. Or due to the fact that the base evaluation, is actually single-threaded, because it expects all the heavy-lifting to be done by the builds.

The phenomenon in this case is called IFD (import from derivation), and is widely used, but nixpkgs, doesn’t like it and banned it from their repo.

All this means is that instead of just nix flake show we need to add the --allow-import-from-derivation flag. After which we get presented with the wonderful: path:/home/... └───packages └───x86_64-linux ├───a: package 'abcl-1.9.2' ├───e: package 'ecl-21.2.1' └───s: package 'sbcl-2.3.8'

YAYYY, our Flake can actually output packages without the need for explicitly writing the nix expression ourselves. And as the builders we run here are arbitrary, there’s no reason why we couldn’t just have a flake, exporting a function that generates all the nix source, from our own system.

Closing remarks

What this means is that it is probably possible, to allow software creators to write a universal, definition which would build in both distros.