Here I'm republishing an old blog post of mine originally from August 2019. The article has been slightly improved.
Installing software on Unix-like systems originally meant compiling everything from source and then copying the resulting program to the desired location. This is much more complicated than people unfamiliar with the subject may think. I've written an article about the history of *nix package management for anyone interested to get an idea of what it once was like and how we got what we have today.
The history of *nix package management [2017 article]
FreeBSD pioneered the _Ports framework_ which quickly spread across the other BSD operating systems that adapted it to their specific needs. Today we have _FreeBSD Ports_, _OpenBSD Ports_ and NetBSD's portable _Pkgsrc_. DragonFly BSD currently still uses _DPorts_ (which is meant to be replaced by Ravenports in the future) and there are a few others like e.g. _mports_ for MidnightBSD. Linux distributions have developed a whole lot of packaging systems, some closer to ports (like Gentoo's _Portage_ where even the name gives you an idea where it comes from), most less close. Ports however are regarded the classical *BSD way of bringing software to your system.
While the ports frameworks have diverged quite a bit over time, they all share a basic purpose: They are used to build binary packages that can then be installed, deinstalled and so on using a package manager. If you're new to package management with FreeBSD's pkg(8), I've written two introduction articles about pkg basics and pkg common usage that might be helpful for you:
FreeBSD package management with Pkg (1/2) [2017 article]
FreeBSD package management with Pkg (2/2) [2017 article]
OpenBSD's ports for example are maintained only to build packages from and the regular user is advised to stay away from using them and to just install pre-built packages. On FreeBSD this is not the case. Using Ports to install software on your machines is nothing uncommon. You will read or hear pretty often that in general people recommend against using ports. They point to the fact that it's more convenient to install packages, that it's so much quicker to do so and that especially updating is much easier. These are all valid points and there are more. You're now waiting for me to make a claim that they are wrong after all, aren't you? No they are right. Use packages whenever possible.
But when it's not possible to use the packages that are provided by the FreeBSD project? Well, then it's best to... still use packages! That statement doesn't make sense? Trust me, it does. We'll come back to it. But to give you more of an idea of the advantages and disadvantages of managing a FreeBSD system with ports, let's just go ahead and do it anyway, shall we?
Just don't do this to a production system but get a freshly installed system to play with (probably in a VM if you don't have the right hardware lying around). It still is very much possible to manage your systems with ports. In fact at work I have several dozen of legacy systems some of which began their career with something like FreeBSD 5.x and have been updated since then. Of course ports were being used to install software on them. Unfortunately machines that old have a tendency to grow very special quirks that make it not feasible to really convert them to something easier to maintain. Anyways... Be glad if you don't have to mess with it. But it's still a good thing to know how you'd do it.
So - when would you use Ports rather than packages? In short: When you need something that packages cannot offer you. Probably you need a feature compiled in that is not activated by default and thus not available in FreeBSD's packages. Perhaps you totally need the latest version in ports and cannot wait for the next package run to complete (which can take a couple of days). Maybe you require software licensed in a way that binary redistribution is prohibited.
Or you've got that 9-STABLE box over there which you know quite well you shouldn't run anymore. But because you relay on special hardware that offers drivers only for FreeBSD 9.x... Yeah, there's always things like that and sometimes it's hard to argue away. But when it comes to packages - since FreeBSD 9 is EOL for quite some time there are no packages being built for it anymore. You'll come up with any number of other reasons why you can't use packages, I'm not having any doubts.
Alright, so let's get to it. You should have a basic understanding of the ports system already. If you haven't, then I'd like to point you to another two articles written as an introduction to ports: Ports basics and building from ports.
FreeBSD: Building software from ports (1/2) [2017 article]
FreeBSD: Building software from ports (2/2) [2017 article]
If you just read the older articles, let me point out two things that have happened since then. In FreeBSD 11.0 a handy new option was added to _portsnap_. If you do
# portsnap auto
it will automatically figure out if it needs to fetch and extract the ports tree (i.e. if it's not there) or if fetching the latest changesets and updating is the right thing to do.
The other thing is a pretty big change that happened to the ports tree in December 2017 not too long after I wrote the articles. The ports framework gained support for _flavors_ and _flavored ports_. This is used heavily for Python ports which can often build for Python 2.7, 3.6, 3.7, ... Now there's generally only one Port for both Python2 and Python3 which can build the py27-foo, py36-foo, py37-foo and so on packages. I originally intended to cover tool-assisted ports management shortly after the last article on Ports, but after that major change it wasn't clear if the old tools would be updated to cope with flavors at all. They were eventually, but since then I never found the time to revisit this topic.
I've setup a fresh FreeBSD 11.2 system on my test machine and selected to install the ports that came with this old version. Yes, I deliberately chose that version, because in a second step we're going to update the system to FreeBSD 11.3.
Using two releases for this has the advantage that it's two fixed points in time that are easy to follow along if you want to. The ports tree changes all the time thanks to the heroic efforts that the porters put into it. Therefor it wouldn't make sense to just use the latest version available because I cannot know what will happen in the future when you, the reader, might want to try it out. Also I wanted to make sure that some specific changes happened in between the two versions of the ports tree so that I can show you something important to watch out for.
There are two utilities that are commonly used to manage ports: _portupgrade_ and _portmaster_. They pretty much do the same thing and you can choose either. I'm going to use portmaster here that I prefer for the simple reason of being more light-weight (portupdate brings in Ruby which I try to avoid if I don't need it for something else). But there's nothing wrong with portupgrade otherwise.
You can of course install portmaster as a package - or you can build it from ports. Since this post is about working with ports, we'll do the latter. Remember how to locate ports if you don't know their origin? Also you can have make mess with a port without changing your shell's pwd to the right directory:
# whereis portmaster
portmaster: /usr/ports/ports-mgmt/portmaster
# make -C /usr/ports/ports-mgmt/portmaster install clean
Portmaster build options (PNG)
As soon as _pkg_ and _dialog4ports_ are built and installed, the build options menu is displayed. Portmaster can be built with completions for bash and zsh, both are installed by default. Once portmaster is installed and available, we can use it to build ports instead of doing so manually.
Building and installing X.org with portmaster (PNG)
Let's say we want to install a minimal X11 environment on this machine. How can we do it? Actually as simple as telling portmaster the origin of the port to install:
# portmaster x11/xorg-minimal
After running that command, you'll find yourself in a familiar situation: Lots and lots of configuration menus for port options will be presented, one after another. Portmaster will recursively let you configure the port and all of its dependencies. In our case you'll have 42 ports to configure before you can go on.
In the background portmaster is already pretty busy. It's gathering information about dependencies and dependencies of dependencies. It starts fetching all the missing distfiles so that the compilation can start right away when it's the particular port's turn. All the nice little things that make building from ports a bit faster and a little less of a hassle.
Portmaster: Summary for building xorg-minimal (1/2) (PNG)
Once it has all the required information gathered and sorted out, portmaster will report to you what it is about to do. In our case it wants to build and install 152 ports to fulfill the task we gave it.
Portmaster: Summary for building xorg-minimal (2/2) (PNG)
Of course it's a good idea to take a look at the list before you acknowledge portmaster's plan.
Portmaster: distfile deletion? (PNG)
Portmaster will ask whether to keep the distfiles after building. This is ok if you're building something small, but when you give it a bigger task, you probably don't want to sit and wait to decide on distfiles before the next port gets built... So let's cancel the job right there by hitting Ctrl-C.
Portmaster: Ctrl-C report (1/2) (PNG)
When you cancel portmaster, it will try to exit cleanly and will also give a report. From that report you can see what has already been done and what still needs to be done (in form of a command that would basically resume building the leftovers).
Portmaster: Ctrl-C report (2/2) (PNG)
It's also good to know that there's the _/tmp/portmasterfail.txt_ file that contains the command should you need it later (e.g. after you closed your terminal). Let's start the building process again, but this time specify the _-D_ argument. This tells portmaster to keep the distfiles. You could also specify _-d_ to delete them if you need the space. Having told portmaster about your decision helps to not interrupt the building all the time.
# portmaster -D x11/xorg-minimal
All right, there we have an error! Things like this happen when working with older ports trees. This one is a somewhat special case. _Texinfo_ depends on one file being fetched that changes over time but doesn't have any version info or such. So portmaster would fetch the current file, but that has silently changed since the distfile information was put into our old ports tree. Fortunately it's nothing too important and we can simply "fix" this by overwriting the file's checksum with the one for the newer file:
# make makesum -C /usr/ports/print/texinfo
# portmaster -D x11/xorg-minimal
Manually fetching missing distfile (PNG)
Next problem: This old version of _mesa_ is no longer available in the places that it used to be. This can be fixed rather easily if you can find the distfile somewhere else on the net! Just put it into its place manually. Usually this is _/usr/ports/distfiles_, but there are ports that use subdirectories like _/usr/ports/distfiles/bash_ or even deeper structures like e.g. _/usr/ports/distfiles/xorg/xserver_!
# fetch https://mesa.freedesktop.org/archive/older-versions/17.x/mesa-17.3.9.tar.xz -o /usr/ports/distfiles/mesa-17.3.9.tar.xz
# portmaster -D x11/xorg-minimal
Eventually the process should end successfully. Portmaster gives a final report - and our requested program has been installed!
Now you know how to build ports using portmaster. There's more to that tool, though. But we'll come to that in the next post which will also show how to use it to update ports.
