Saturday, 05. April 2021
[This article has been bi-posted to Gemini and the Web]
In this article we'll continue exploring Synth. After covering a general introduction to package building as well as Synth's basic operation in the previous articles, we're going to look at the program's configuration and using it for updating the system.
FreeBSD package building pt. 1: Introduction and test system
FreeBSD package building pt. 2: Basic Synth
The first thing to do here is to get rid of the old ports tree and replace it with a newer one so that some updates become available:
# zfs destroy zroot/usr/ports
# zfs create zroot/usr/ports
# git clone --depth 1 -b 2012Q2 https://git.freebsd.org/ports.git /usr/ports
# mkdir /usr/ports/distfiles
# synth status
Synth regenerates the flavor index again, then gives the output. This time it will not just show all packages as "new" but most are either updates or rebuilds.
Synth status with new ports tree (PNG)
Let's look at how you can change Synth's behavior next:
# synth configure
Synth's configuration is pretty straight-forward. Options _A_ to _G_ configure the various directories that the program uses. Should you change anything here? If you know what you are doing and have special needs: Maybe. If you actually ask the question whether to change any such directory, the answer is _no_. Option _H_ (Compiler cache) is a special one. We'll get to that in a minute.
With _I_ you set the number of builders that Synth uses. _J_ means how many threads each builder may use; think _make -j $SOMENUMBER_. When compiling a single piece of software, it's usually recommended to set the number of jobs equal to the machines core count + 1. Take a look at the screenshot above: Doing the math, we're configuring for _24_ cores here - on a system that has _8_ (with Hyper Threading).
Synth's configuration menu (PNG)
Why does Synth choose to over-provision the resources so much? The reason is simple: It's only over-provisioned when more than three builders are active at the same time. Often enough not all builders will be at the build stage (where the compiling happens) at the same time. Most other stages are much lighter on CPU - which would mean that you're wasting available resources (and thus prolong the total build time). Also in the previous post you've seen that LLVM and Rust took hours to build (all four remaining builders were _idle_ most of the time!). If the cap would have been lower, build times would have increased even more.
So what's the best setting for builder count and max jobs? There's no single answer to that. It depends on both your machine and on the set of ports that you build. Play with both values a bit and get a feeling what works best for you. Or leave it at the default that Synth calculated for your system which will probably work well enough.
Options _K_ and _L_ are real speed boosters. They control whether the base system for the builder ("localbase") and the directory where the program is built are using _tmpfs_ or not. Tmpfs is a memory-backed filesystem. If you disable one or both options, compilation times increase _a lot_ because all the data that needs to be copied on setting up a builder and compiling software will be written to your disk. For an extreme example: On one of my machines, building and testing a port increased from slightly over 30 seconds with tmpfs to over 4 minutes (!) without it. Yes, that machine uses an HDD and it was occupied with other things besides building packages. But there is more than a negligible impact if you disable tmpfs.
So when to do it in the first place? If you're on a machine with little RAM you might have to do this. Some ports like LLMV or Firefox require _lots_ of RAM to build. If your system starts swapping heavily, disable tmpfs. Ideally build those ports separately and leave tmpfs on for all the rest.
Then there's option _M_ which toggles the fancy colored text UI on or off. If you turn it off, you'll get a simple monochrome log-like output of which builder started or finished building which port. Every now and then the info that's in the top bar of the UI (e.g. number of packages completed, number of packages remaining, skips, etc) gets printed.
Finally we have option _N_ which toggles using pre-built packages on or off. It's off by default which means that Synth will build everything required for the selected package set. If you enable this option and have e.g. the official FreeBSD repository enabled as well, it will try to fetch suitable packages from there that can be used as the buildtime or runtime dependencies of the packages that still need to be built. If you're mixing official and custom packages this could be a huge time saver. I admit that I have never used this option.
And what's this _profile_ thing? Well, ok. The configuration we've been looking at is for the _default profile_. You can create additional ones if you want to. And that's where the directories that I told you to ignore come into play. You could for example create a profile to use a different ports tree (e.g. the _quaterly_ branch) and switch between the profiles to build two different packages sets on one machine. While Synth _can_ do this, that is the point where I'd advise you to try out Poudriere instead. The main benefit of Synth over Poudriere is ease of use and when you're trying to do clearly advanced things with Synth you might as well go for the officially supported FreeBSD package builder instead.
Let's disable the curses UI just to see what Synth looks like without it and save the config change. If you plan to build packages regularly on your system, you will definitely want to set up the _compiler cache_. To be able to use it, we first need another package installed, though: _ccache_. We're going to build it and then install it manually this time:
# synth just-build devel/ccache
# pkg add /var/synth/live_packages/All/ccache-3.7.1_1.txz
Then we're going to create a directory for it and go back to Synth's configuration menu:
# mkdir -p /var/tmp/ccache/synth
# synth configure
Now change _H_ to point to the newly created directory and save. Synth will use ccache now. But what does it do? Ccache caches the results from compilation processes. It also detects if the same compilation is happening again and can provide the cached result instead of actually compiling once again. This means that the first time you compile it doesn't make a difference but after that the duration of building packages will drop _significantly_. The only cost of this is that the cached results take up a bit of space. Keep ccache disabled if drive space is your primary concern. In all other cases definitely turn it on!
Ccache directory and config after the system upgrade (PNG)
Next is using Synth to update the system.
# synth upgrade-system
After determining which packages need to be built / rebuilt, Synth will start doing so. We've turned off the text UI, so now we get only the pretty simplistic output from standard text mode.
Package building in pure text mode (PNG)
It provides you with the most important information about builders: Which builder started / finished building which package when. You don't get the nice additional info about which state it's in and how long it has been busy building so far. As mentioned above, Synth will print a status line every couple of minutes which holds the most important information. But that's all.
Status lines amidst the builder output (PNG)
What happens if something goes wrong? I simulated that by simply killing one of the processes associated with the rust builder. Synth reports failure to build rust and prints a status line that shows 17 package skips. In contrast to the curses UI it does not tell you explicitly which ones were skipped, though!
Simulated package build failure (PNG)
When Synth is done building packages, it displays the tally as usual and does some repository cleanup by removing the old packages. Then it rebuilds the repository.
Tally displayed after completion and repository cleanup (PNG)
Since we asked Synth to upgrade the system, it invokes pkg(8) to do its thing once the repository rebuild is complete.
Package repository rebuilt, starting system upgrade (PNG)
And here's why I strongly prefer _prepare-system_ over _upgrade-system_: The upgrade is initiated whether there were failed packages or not. And since pkg(8) knows no mercy on currently installed programs when they block upgrades, it will happily remove them by default! To be safe it makes sense to always review what pkg(8) would do before actually letting it do it. Yes, it's an additional step. Yes, most of the time you'll be fine with letting Synth handle things. But it might also bite you. You've been warned.
System upgrade in progress - it may remove packages! (PNG)
Every now and then you may want to run _synth purge-distfiles_ (unless you have unlimited storage capacity, of course). It will make the tool scan the distinfo files of all ports and then look for distfile archives of obsolete program versions to remove.
Cleaning up old distfiles with purge-distfiles (PNG)
There might not be a large gain in our example case, but things do add up. I've had Synth reclaim multiple GB on a desktop machine that I regularly upgraded by building custom packages. And that's definitely worth it.
The next article will cover some leftover topics like logs, web report and repository sharing.
