Originally in the design of BioNix a choice to implement cluster integration at the level of Nix was made to simplify implementation and maximise reuse of existing tools (the Nix builder). This has lead to a few downsides:
1. only BioNix stages are submitted to the cluster, software building happens directly on the machine running nix;
2. derivations capture platform specific configuration, making it difficult to form ad-hoc remote builders.
The latter prevents forming buildfarms comprising of multiple HPC facilities, as derivations cannot easily pass between them. The solution to these issues is to push Slurm support down into the Nix build core: by submitting builds to the scheduler instead of executing them directly, HPC resources managed through a scheduling system just becomes a specific type of builder.
A proof of concept is avilable:
Here it is building some python dependencies in order to do an analysis:
JOBID PARTITION NAME USER STATE TIME TIME_LIMI NODES
2970825 regular pytz-2021.3-py2.py3-none-any.whl bedo.j PENDING 0:00 2-00:00:00 1
2970817 regular python3.8-kiwisolver-1.3.1 bedo.j RUNNING 1:22 2-00:00:00 1
2970818 regular python3.8-pyparsing-3.0.3 bedo.j RUNNING 1:22 2-00:00:00 1
2970809 regular python3.8-joblib-1.1.0 bedo.j RUNNING 1:54 2-00:00:00 1
2970810 regular python3.8-numpy-1.20.2 bedo.j RUNNING 1:54 2-00:00:00 1
2970811 regular python3.8-pillow-8.2.0 bedo.j RUNNING 1:54 2-00:00:00 1
2970812 regular python3.8-psutil-5.8.0 bedo.j RUNNING 1:54 2-00:00:00 1
2970814 regular python3.8-xlrd-1.2.0 bedo.j RUNNING 1:54 2-00:00:00 1