Here I'm republishing an old blog post of mine originally from September 2019. The article has been slightly improved.
In the previous post I explained why sometimes building your software from ports may make sense on FreeBSD. I also introduced the reader to the old-fashioned way of using tools to make working with ports a bit more convenient. The article also included links to other posts about package management on FreeBSD as well as an introduction to ports that I'll not put here again.
Using FreeBSD with Ports (1/2): Classic way with tools
In this follow-up post we're going to take a closer look at _portmaster_ and see how it especially makes updating from ports much, much easier. For people coming here without having read the previous article: What I describe here is *not* what every FreeBSD admin today should consider good practice (any more)! It can still be useful in special cases, but my main intention is to discuss this for building up the foundation for what you actually should do today.
Last time we stopped after installing the _xorg-minimal_ meta-port. Let's say we want a very simple desktop installed on this machine. I chose the most frugal *nix desktop, EDE (the Equinox Desktop Environment, looking kind of like Win95), because that's drawing in things that I need for demonstrating a few interesting things here - and not _that much more_.
Unfortunately in the ports tree that we're using, exactly that port is broken (the newer compiler in FreeBSD 11.2 is more picky than the older ones and not quite happy with EDE code). So to go on with it, we have to fix it first. I've uploaded an additional patch file from a later version of the port and also prepared a patch for the port's Makefile. If you want to follow along, you can just copy the three lines below to your terminal:
# fetch http://www.elderlinux.org/files/patch-evoke_Xsm.cpp -o /usr/ports/x11-wm/ede/files/patch-evoke_Xsm.cpp
# fetch http://www.elderlinux.org/files/patch_ede_port -o /usr/ports/x11-wm/ede/patch_ede_port
# patch -d /usr/ports/x11-wm/ede -i patch_ede_port
Using Portmaster to build and install EDE (PNG)
Thanks to build-time dependencies and default options in FreeBSD it's still another 110 ports to build, but that's fine. We could remove some unneeded options and cut it down quite a bit. Just to give you an idea: By configuring only _one_ package (doxygen) to not pull in all the dependencies that it usually does, it would be only 55 (!) ports.
But let's say we're lazy. Do we have to face all of those configure dialogs (72 in cause you are curious)? No, we don't. That's why portmaster has the _-G_ flag which skips the config make target and just uses the standard port options:
# portmaster -DG x11-wm/ede
EDE was successfully installed (PNG)
Using this option can be a huge time-saver if you're building something where you know that you don't need to change the options for the application and its dependencies.
Now that we have a simple test system with 265 installed but outdated packages. Let's update it! Remember that unlike e.g. Linux, FreeBSD keeps third party software installed from packages or ports and the actual operating system separate. We'll update the latter first:
# freebsd-update upgrade -r 11.3-RELEASE
With this command, we make the updater download the required files for the upgrade from 11.2-RELEASE to 11.3-RELEASE.
Upgrading FreeBSD to 11.3-RELEASE (PNG)
When it's done, and we've closed the three lists with the removed, updated and new files, we can install the new kernel:
# freebsd-update install
Once that's done, it's time to reboot the system so the new kernel boots up. Then the userland part of the operating system is installed by using the same command again:
# shutdown -r now
# freebsd-update install
Now in our fresh 11.3 system we should first get rid of the old ports tree to replace it with a newer one, right? Wait, hold that _rm_ command for a second and let me show you something really useful!
If you take a look at the _/usr/ports_ directory, you'll find a file appropriately named _UPDATING_. And since that's right what we were about to do, why not take a look at it?
So what is this? It's an important file for people updating their systems using ports. Here is where ports maintainers document breaking changes. You are free to ignore it and the advice that it gives and there's actually a chance that you'll get away with it and everything will be fine. But sometimes you don't - and fixing stuff that you screwed up might take you quite a bit longer than at least skimming over UPDATING.
# less /usr/ports/UPDATING
But right now it's completely sufficient to look at the metadata of the first notification which reads:
20180330:
AFFECTS: users of lang/perl5*
AUTHOR: mat@FreeBSD.org
The main takeaway here is the date. The last heads-up notice for our old ports tree was on _2018-03-30_.
Checking out a newer ports tree (PNG)
Now let's throw it all away and then get the new ports tree. Usually I'd use _portsnap_ for this, but in this case I want a special ports tree (the one that would have come with the OS if I got ports from a fresh 11.3 installation), so I'm checking it out from SVN:
# rm -rf /usr/ports/.* /usr/ports/*
# svnlite co svn://svn.freebsd.org/ports/tags/RELEASE_11_3_0 /usr/ports
If you're serious about updating a production server that you care about, now is the time to read through _UPDATING_ again. Search for the date string that you previously took a note of and then read the messages all the way up to the beginning of the file. It's enough to read the AFFECTS lines until you hit one message that describes a port which you are using. You can ignore all the rest but should really read those heads-up messages that do affect your system.
What software can be updated? (PNG)
BTW... You know which packages you have installed, don't you? A huge update like what we're facing here takes some planning up-front if you want to do it in a professional manner. In general you should update way more often, of course! This makes things much, much easier.
Ok, we're all set. But which software can be updated? You can ask pkg(8) to compare installed packages to the respective distinfo from the corresponding port:
# pkg version -l "<"
If you pipe that into wc -l you will see that 165 of the 265 installed packages can (and probably should) be updated.
Updating software from ports (PNG)
We'll start with something really simple. Let's say, we just want to update _pkgconf_ for now. How do we do that with portmaster? Easy enough: Just like we would have portmaster install it in the first place. If something is already installed, the tool will figure out if an update is available and then offer to update it.
# portmaster -G devel/pkgconf
And what will happen if the port is already up to date? Well, then portmaster will simply re-build and re-install it.
While partial updates are possible, it's a much better idea to keep the whole system updated (if at all possible). To do that, you can use the _-a_ flag for portmaster to update all installed software.
Another nice flag is _-i_, which is for _interactive mode_. In this mode, portmaster will ask for every port if it should be updated or not. If you're leaving out critical ports (dependencies) this can lead to an impossible update plan and portmaster will not start updating. But it can be useful for cherry-picking.
Now let's attempt to upgrade all the ports, shall we?
# portmaster -aG
As always, portmaster will show you its plan and ask for your confirmation. There are two things that probably deserve a word or two about them. Usually the plan is to update an application, but sometimes portmaster wants to _re-install_ or even _downgrade_ them (see picture)! What's happening here?
Re-installs mostly happen when a dependency changed and portmaster figured out that it might be a good idea to rebuild the port against the newer version of the dependency, even though there is no newer version available for the actual application.
Upgrading, "downgrading", re-installing (PNG)
Downgrades are a different thing. They can happen when you installed something from a newer ports tree and then go back to an older one (something you usually shouldn't do). But in this case it's actually a false claim. Portmaster will not downgrade the package - it was merely confused by the fact that the versioning scheme changed (because of the 0 in 2018.4 it thinks that this version is older than the previous 2.1.3...).
If you're paying close attention to all the information that portmaster gives you, you'll have seen lines like the following one:
===>>> The x11/bigreqsproto port moved to x11/xorgproto
There's another interesting file in the ports tree called _MOVED_. It keeps track of moved or removed ports. Sometimes ports are renamed or moved to another category if the maintainer decides it fits better. Portmaster for example started as sysutils/portmaster and was moved later when the ports-mgmt category was introduced. However you won't find this information in the MOVED file - because it happened before the time that the current MOVED keeps records for (i.e. early 2007 in this case). 😉
The example above is due to the fact that last year the upstream project (Xorg) decided to combine the protocol headers into one distribution package. Before that there were more than 20 separate packages for them (and before that, once upon a time, all of Xorg had been one giant monolithic release - but I digress...)
Problem with merged ports (PNG)
The good news here is that portmaster is smart enough to parse the MOVED file and figure out how to deal with this kind of changes in the ports tree! The bad news is that this does not work for more complicated things like the merges that we just talked about...
So what now? Good thing you read the relevant UPDATING notification, eh?
20180731:
AFFECTS: users of x11/xorg and all ports with USE_XORG=*proto
AUTHOR: zeising@FreeBSD.org
Bulk-deleting obsolete ports and trying again (PNG)
So let's first get rid of the old *proto packages with the command that developer Niclas Zeising proposes and then try again:
# pkg version -l \? | cut -f 1 -w | grep -v compat | xargs pkg delete -fy
# portmaster -aG
Alright, we have one more problem to overcome. There are ports that will fail to build if we run portmaster with the -G flag. Why? Because they have _mandatory ports options_ where you need to choose from things like a backend or a certain mechanic.
The "mandatory options" case (PNG)
One such case is _freetype2_. Since it fails, build it separately and do not skip the config dialog for this one:
# portmaster print/freetype2
Once that's done, we can continue with updating all the remaining ports. After quite a while (because LLVM is a beast) all should be done!
Did you read the following notice in UPDATING?
20181213:
AFFECTS: users of lang/perl5*
AUTHOR: mat@FreeBSD.org
For the big update run we ignored this. And in fact, portmaster _did_ update Perl, but only to the latest version in the 5.26 branch of the language:
# pkg info -x perl
perl5-5.26.3
Why? Well, because that was the version of Perl that was already installed. Actually this is ok, if you're not using Perl yourself and thus can live without the latest features added. However if you want to (or have to) upgrade to a later Perl major version we have a little bit more work to do.
First edit _/etc/make.conf_ and put the following line in there:
DEFAULT_VERSIONS+=perl5=5.28
This is a hint to the ports framework that whenever you're building a Perl port, you want to build it against this version. If you don't do that, you will receive a warning when building the other Perl. Because in this case you're installing an additional Perl version but all the ports will use the primary one. So more likely than not you don't want to do this.
Next we need to build the newer Perl. The special thing here is that we need to tell portmaster of the changed origin so that the new version actually replaces the old one. We can do this by using the _-o_ flag. Mind the syntax here, it's _new origin first_ and _then old origin_ and not vice versa (took me a while to get used to it...)!
But let's check the origin real quick, before we go on. The pkg command above showed that the package is called _perl5_. This outputs what we wanted to know:
# pkg info perl5
perl5-5.26.3
Name : perl5
Version : 5.26.3
Installed on : Tue Sep 10 21:15:36 2019 CEST
Origin : lang/perl5.26
[...]
There we have it. Now portmaster can begin doing its thing:
# portmaster -oG lang/perl5.28 lang/perl5.26
Rebuilding ports that depend on Perl (PNG)
Ok, the default Perl version has been updated... But this broke all the Perl ports! So it's time to fix them by rebuilding against the newer Perl 5.28. Luckily the UPDATING notice points us to a simple way to do this:
# portmaster -f `pkg shlib -qR libperl.so.5.26`
And that's it! Congratulations on updating an old test system via ports.
You know your system and applications, are proficient with your shell and you've read UPDATING. What could possibly go wrong? Well, we're dealing with computers here. Something really strange can go wrong anytime. It's not very likely, but sometimes _things happen_.
Portmaster can help you if you ask for it _before_ attempting upgrades. Before deinstalling an old package, it creates a backup. However after installing the new version it throws it away. But you can make it keep the backup by supplying the _-b_ flag. Sometimes the old package can come in handy if something goes completely sideways. If you need backup packages, have a look in _/usr/ports/packages/portmaster-backup_. You can simply _pkg add_ those if you need the old version back (of course you need to be sure that you didn't update the packages dependencies or you need the downgrade them again, too!).
If you want to be extra cautious when updating that very special old box (that nobody dared to touch for nearly a decade because the boss threatened to call down terrible curses (not the library!) upon the one who breaks it), portmaster will also support you. Use the _-w_ flag and have it preserve old shared libs before deinstalling an old package. I wouldn't normally do it (and think my example made it clear that it's really special). In fact I've never used it. But it might be good to know about it should you ever need it.
That said, on the more complicated boxes I usually create a temporary directory and issue a _pkg create -a_, completely backing up all the packages before I begin the update process. Usually I can throw away everything a while later, but having the backups saved me some pain a couple of times already.
In the end it boils down to: Letting your colleagues call you a coward or being the tough guy but maybe ruining your evening / weekend. Your choice.
Anf if you need to know even more about the tool that we've been using over and over now, just _man portmaster_.
I haven't decided on the next topic that I'm going to write about. However I've planned for two more articles that will cover building from ports _the modern way(tm)_ - and I hope that it will not take me another two years before I come to it...
