💾 Archived View for gemini.circumlunar.space › users › kraileth › neunix › 2021 › freebsd_package_bu… captured on 2022-04-28 at 19:10:12. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2021-12-05)
-=-=-=-=-=-=-
Tuesday, 25. May 2021
[This article has been bi-posted to Gemini and the Web]
In 2017 I started to post a series of well-received articles that was meant to culminate in a tutorial on building packages on FreeBSD. I wrote about the history of package managers, about how to use FreeBSD's pkg(8) package manager as well as an introduction to working with ports.
The history of *nix package management [2017 article]
FreeBSD package management with Pkg (1/2) [2017 article]
FreeBSD package management with Pkg (2/2) [2017 article]
FreeBSD: Building software from ports (1/2) [2017 article]
FreeBSD: Building software from ports (2/2) [2017 article]
Then I had to stop for technical reasons (after a major change to the ports infrastructure, the tool that I wanted to write about had not been updated to work with the new ports tree, yet!). In 2019 I eventually found the time to publish two more articles that were originally meant to come right after the 2017 posts. They covered using classic tools for ports management.
Using FreeBSD with Ports (1/2): Classic way with tools [2019 article]
Using FreeBSD with Ports (2/2): Tool-assisted updating [2019 article]
They were meant to be the stepping stone to what I _actually_ wanted to cover: Package building! The second article ended with:
I've planned for two more articles that will cover building from ports _the modern way(tm)_ - and I hope that it will not take me another two years before I come to it...
That was... Yikes! In fall 2019... So I'd better hurry up now. I had hinted that using ports directly was not the recommended thing to do anymore and that while you should use packages unless you need ports, you really should use packages even in the latter case! And that's what we're going to do in the next few articles.
There are valid reasons to not use FreeBSD's official packages:
Things like that. People choose ports when they _need more control_ over their applications. There are good reasons to _avoid using ports the traditional way_, too, however:
While the former two points are mostly relevant if you manage multiple machines, I'd recommend rolling your own packages even for the single FreeBSD workstation that you use - _if the official packages don't suit you_. Let me state this again: Try to go with the packages provided by the FreeBSD project first. Build your own packages if you have to (educating yourself is a completely valid reason to do it anyway, though).
When you decide to roll your own packages, you have two options: *Synth*, the _much_ easier and nicer package builder and *Poudriere*, the advanced build tool that FreeBSD uses and provides official documentation for.
Which one should you choose? I'm going to show how to work with both so you can compare them and make an informed decision. If you're just getting started, you may want to give Synth a try. It is also a good choice when you use DragonFly BSD, too: The _dsynth_ tool that they have in base was inspired by Synth (and if it wasn't written in Ada they certainly would just have imported it instead of creating a re-implementation in C). You should also know that the Synth project is in maintenance mode. Its author still fixes bugs and takes pull requests on GitHub, but it's feature-complete.
The main advantage of Synth is that it's dead simple to setup and use, too, whereas Poudriere is a somewhat complex beast. Synth also shines when you want to use it to keep one machine up to date with packages as it can do those updates for you. Poudriere on the other hand allows you to do things like maintaining package repositories for multiple versions of FreeBSD as well as multiple architectures from one build machine. If you need that, forget Synth.
One major change that was made in FreeBSD since the previous article was published is that the project migrated to using Git repositories. FreeBSD development started on CVS but in 2008 the _src_ repository was successfully migrated to using Subversion. In mid 2012 _docs_ and _ports_ also made the switch. Subversion has been used ever since until December 2020 when _doc_ and _src_ transitioned to Git. After some remaining issues were solved, _ports_ also migrated to Git in April 2021. While src changes get pushed back to Subversion for FreeBSD's branches of 11 and 12, when it comes to ports, an era has ended.
Get rid of that habit of visiting _svnweb.freebsd.org_ and start getting used to working with _cgit.freebsd.org_ instead:
FreeBSD cgit web repository overview
If you are unsure of your Git skills, you may want to at least skim over the _Git-primer_ in FreeBSD's documentation:
At least get familiar with the concepts. You should for example know that Git is able to do things like a _shallow clone_; looking things up when you need them is no problem but not being aware of them at all _is_.
While both Subversion and Git are used for version control and both use repositories, they are fundamentally different. Subversion is the best known version control system of the second generation (so-called (centralized) "networked VCS"). Git is the most popular one of the third generation ("decentralized VCS").
If you use Subversion, there's a central repository somewhere and you _checkout_ a certain revision of the versioned files from there. The files that were checked out form your _working directory_. In Git you _clone_ a remote repository and thus receive all the data needed to get a local copy of the whole repo. The files for the working directory are checked out from the local copy. The major difference is: You have the _whole history_ available locally.
In case you have no need for all that data, you do a _shallow clone_ instead of a regular full clone. To give you an example: If you do a shallow clone of the current ports tree today, the result is about 840 MB in /usr/ports - of which 85 MB in size is the Git repository. A full clone is about 1.7 GB in size of which about 920 MB is for the repo! So if you don't need the history, save some space on your drive and save some donated bandwidth of the FreeBSD project.
While you can certainly start rolling your own packages on a fresh system, it's also fine to begin doing so on a system that has used the standard FreeBSD ports or packages so far. There's nothing wrong with that actually. Taking the opposite way and going back from custom packages to the official ones is also possible, of course. That latter case requires some more planning, though. Think about why you started building your own packages in the first place, find out if the standard packages fit your current needs and think about the consequences. If you're uncertain, you may want to start over with regular packages. If you want to make the switch back, it's best to re-install all the packages (use the flag _-f_ with _pkg upgrade_). If you're using ZFS in a way that supports Boot Environments, create one first.
In fact you can even choose to use both official and custom packages on the same machine: Instead of building the full set of packages that you need, you build just the packages yourself that you need to customize and use the packages from the standard FreeBSD package repositories for the rest. This works by configuring your package repository as an additional one for pkg(8) and giving it a higher priority. If you consider doing this and using multiple package repositories, be sure to
% man 5 pkg.conf
first. It doesn't hurt to skim over the whole manpage, but make sure that you at least read the section REPOSITORY CONFIGURATION. I used that approach on a weaker machine in the past, but wouldn't generally recommend it. If your hardware is recent enough, you can compile everything yourself. Otherwise it might make more sense to build on a somewhat beefy system and distribute the packages to your other machine(s). But that's only me. Maybe mixing packages is the right solution for your use case.
The rest of this post is about building a test system to simulate an environment with pre-installed packages. I do this so that I can show off a few things regarding updates in the next post. If you want to follow along, make sure that you have Git installed and that your directory or dataset /usr/ports as well as /usr/src is _empty_. I'm assuming that you're using FreeBSD 13.0.
The first thing to do is getting the ports tree in place. I'm doing a shallow clone of the 2021 first quarter branch (so I can update later). Then I clone operating system source repository as well:
# git clone --depth 1 -b 2021Q1 https://git.freebsd.org/ports.git /usr/ports
# git clone --depth 1 -b releng/13.0 https://git.freebsd.org/src.git /usr/src
Now we don't need the packages anymore and forcefully remove them all (including pkg). The reason for this is that we want to build the older versions of the old ports tree that we just cloned.
# pkg delete -af
For convenience we're going to build all the required programs using portmaster as discussed in a previous article (see the beginning of this post). Of course we need to build and install it first:
# make -C /usr/ports/ports-mgmt/portmaster install clean
Alright. Now we're building some leaf ports to simulate a very simple development system. These ports draw in a number of dependencies so that we end up with about 350 packages (which is still a *very* low count for a desktop system). I'm building _tmux_ and _sudo_ simply because I *always* need to have them available and git just because we needed it before anyway. The rest is the _graphics drivers_, a simple subset of _X11_ + the command to set a different keyboard layout as well as the _awesome window manager_ and the simple but nice GTK+ terminal emulator called _sakura_. Since I'll need to take screenshots for the upcoming posts, I figured that I might include an application for that, too.
# portmaster -dG sysutils/tmux security/sudo devel/git graphics/drm-kmod x11/xorg-minimal x11/setxkbmap x11-wm/awesome x11/sakura x11/xfce4-screenshooter-plugin
And that's it for today.
Next station is taking a look at building packages and updating the system using _Synth_.