Here I'm republishing an old blog post of mine originally from September 2017. The article has been slightly improved.
Please mind that there have been some changes in FreeBSD after the article was written. Two important ones that you should be aware of are: The introduction of so-called "flavored ports" (I covered these when revisiting the topic two years after writing this one) and the transition from Subversion to Git.
In my previous three posts I wrote about a short history of *nix package management and then about using Pkg, FreeBSD's package manager.
The history of *nix package management
FreeBSD package management with Pkg (1/2)
FreeBSD package management with Pkg (2/2)
Pre-built binary packages are convenient to use but sometimes you need some more flexibility, want an application that cannot not be distributed in binary form due to license issues (or have some other requirements). Building software by hand is certainly possible - but with all the things involved, this can be a rather tedious process. It's also slow, error-prone and there's often no clean way to get rid of that stuff again. FreeBSD Ports to the rescue!
This first part is meant as a gentle introduction to FreeBSD's ports, assuming no prior knowledge (if I fail to explain something, feel free to let me know). It will give you enough background information to understand ports enough to start using them in the next article.
When programmers talk about _porting something_ over, what they originally meant is this: Take an application that was written with one processor architecture in mind (say i386) and modify the source so that it runs on another (arm64 for example) afterwards. The term "porting" is also used when modifying the source of any program to make it run on another OS. The version that runs on the other architecture / OS is called _a port_ of the original program to a different platform.
FreeBSD uses the term slightly differently. There's a lot of software written e.g. for Linux that will build and work on FreeBSD just fine as it is. Even though it does not require any changes, that software might be *ported* to FreeBSD. So in this case "porting" does not mean "make it work at all" but _make it easily available_. This is done by creating a *port* for any program. That term doesn't mean a variant of the source code in this case but rather a means to give you easy access to that software on FreeBSD.
So what is a port in FreeBSD? Actually a port is a directory with a bunch of files in it. The heart of it is one file that basically is a recipe if you will. That recipe contains everything needed to build and install the port (and thus have the application installed on your machine in the end). Following this metaphor you could think of all the ports as a big cookbook. Formally it is known as *the Ports collection*. All those files in your filesystem related to ports are refered to as *the Ports tree*.
There are several options to obtain a copy of the ports tree. When you install FreeBSD you can decide whether or not to install it, too. I usually don't do that because on systems that use binary packages only. It wastes only about 300 MB of space, but more importantly consists of almost 170.000 files (watch your inodes on embedded devices!). Take a look at _/usr/ports_: If that directory is empty your system is currently missing the ports tree.
The simplest way to get it is by using _portsnap(8)_:
# portsnap fetch extract
If you want to update the tree later, you can use:
# portsnap fetch update
Another way is to use Subversion. This is more flexible: With portsnap you always get the current tree while Subversion also allows you to checkout older revisions, too. If you plan to become a ports developer, you will probably want to use Subversion for tools like _svndiff_. If you just want to use ports, portsnap should actually suffice. All currently supported versions of FreeBSD contain a light-weight version of Subversion called _svnlite_.
Here's how to checkout the latest tree:
# svnlite checkout https://svn.freebsd.org/ports/head /usr/ports
If you want to update it later run:
# svnlite update /usr/ports
You normally shouldn't need these but it's good to know that they exist. Using Subversion you can also retrieve old trees. Be sure that _/usr/ports_ is empty (including for Subversion's dot directories) or Subversion will see that there's already something there and won't do the checkout. If for example you want the ports tree as it existed in 2016Q4, you can retrieve it like this:
# svnlite checkout https://svn.freebsd.org/ports/branches/2016Q4 /usr/ports
There are also several tags available that allow to get certain trees. Maybe you want to see which ports were available when FreeBSD 9.2 was released. Get the tree like this:
# svnlite checkout https://svn.freebsd.org/ports/tags/RELEASE_9_2_0 /usr/ports
And if you need the last tree that is guaranteed to work with 9.x there's another special tag for it:
# svnlite checkout https://svn.freebsd.org/ports/tags/RELEASE_9_EOL /usr/ports
Keep in mind though that using old trees is risky because they contain program versions with vulnerabilities that have since been found! Also mind that it's NOT a smart thing to simply get the tree for RELEASE_7_EOL because it still holds a port for PHP 5.2 and you thought that it would be cool to offer your customers as many versions as possible. Yes, it may be possible that you can still build it if you invest some manual work. But no, that doesn't make it a good idea at all.
Oh, and don't assume that old ports trees will be of any use on modern versions of FreeBSD! The ports architecture changed quite a bit over time, the most notable change being the replacement of the old pkg_* tools with the new Pkg. Ports older than a certain point in time definitely won't build in their old, unmodified state today (and I say it again: You really shouldn't bother unless you have a very special case).
Take a look at the contents of /usr/ports on a system that has the tree installed. You will find over 60 directories there. There are a few special ones like _distfiles_ (where tarballs with program's source code get stored - might be missing initially) or _Mk_ that holds include files for the ports infrastructure. The others are categories.
If you're looking for a port for Firefox, that will be in _www_. GIMP is in _graphics_ and it's probably no surprise that Audacious (a music player) can be found in _audio_. Some program's categories will be less obvious, however. LibreOffice is in _editors_ which is not so bad. But help2man for example is in _misc_ and not in _converters_ or _devel_ as at least I would expect if I didn't know. In general however after a while of working with ports you will have a pretty good chance to guess right where things are.
Say we are interested in the port for the window manager Sawfish for example. It's located in _/usr/ports/x11-wm/sawfish_. Let's take a closer look at that location and take it apart:
_/usr/ports_ is the "ports directory".
_x11-wm_ (short for X11 window managers) is the *category*.
_sawfish_ is the individual port's *name*.
When referring to where a port lives, you can omit the ports directory since everybody is assumed to know where it is. The important information when identifying a port is the category and the name. Together those form what is known as the *port origin* (_x11-wm/sawfish_ in this example).
There are multiple methods to find out the origin for the port you are looking for. Probably the simplest one is using _whereis(1)_. If we didn't know that sawfish is in x11-wm/sawfish we could do this:
% whereis sawfish
sawfish: /usr/ports/x11-wm/sawfish
This does however only work if you know the exact name of the port. And there's a little more to it: Sometimes the name of a port and the package differ! This is often the case for Python-based packages. I have SaltStack installed, for example. It's a package called _py27-salt_:
% pkg info -x salt
py27-salt-2017.7.1_1
If we were to look for that, we wouldn't find it:
% whereis py27-salt
py27-salt:
So where is the port for the package?
% pkg info py27-salt
py27-salt-2017.7.1_1
Name : py27-salt
[...]
Origin : sysutils/py-salt
[...]
Here you can see that the port's name is _py-salt_! The "27" gets added when the package is created and reflects the version of Python that it's build against. You may also see some _py3-xyz_ ports. In those cases the name reflects that the port cannot be built with Python 2.x. The package will still be called _py36-xyz_, though (or whatever the default Python 3.x version is at that time)!
When discussing package management I recommended FreshPorts and when working with ports it can be useful, too. Search for some program's name and it might be easier for you to find the package name and the port origin for it!
Let's take a look at the port for the _zstd_ compression utility:
% ls /usr/ports/archivers/zstd/
distinfo Makefile pkg-descr pkg-plist
So what do we have here? The simplest file is _pkg-descr_. Each package has a short and a long package description - this file is what contains the latter: A detailed description that should give you a good idea whether this port would satisfy your needs:
% cat /usr/ports/archivers/zstd/pkg-descr
Zstd, short for Zstandard, is a real-time compression algorithm providing
high compression ratios. It offers a very wide range of compression vs.
speed trade-offs while being backed by a very fast decoder. It offers
[...]
Then there's a file called _distinfo_. It lists all files that need to be downloaded to build the port (usually the program's source code). It also contains a checksum and the file's size to make sure that the valid file is being used (an archive could get corrupted during the transfer or you could even get an archive that somebody tempered with!):
% cat /usr/ports/archivers/zstd/distinfo
TIMESTAMP = 1503324578
SHA256 (facebook-zstd-v1.3.1_GH0.tar.gz) = 312fb9dc75668addbc9c8f33c7fa198b0fc965c576386b8451397e06256eadc6
SIZE (facebook-zstd-v1.3.1_GH0.tar.gz) = 1513767
There's usually also the _pkg-plist_. It lists all the files that are installed by the port:
% cat /usr/ports/archivers/zstd/pkg-plist
bin/unzstd
bin/zstd
bin/zstdcat
[...]
lib/libzstd.so.%%PORTVERSION%%
libdata/pkgconfig/libzstd.pc
man/man1/unzstd.1.gz
man/man1/zstd.1.gz
man/man1/zstdcat.1.gz
And finally there's the _Makefile_. This is where all the magic happens. If you're a programmer or you have built software from source before, there's a high chance that you're at least somewhat familiar with a tool called _make_. It processes Makefiles and then does as told by those. While it's most often used to compile software it can actually be used for a wide variety of tasks.
If you don't have at least some experience with them, Makefiles look pretty much obscure and creating them seems like a black art. If you've ever looked at a complicated Makefile, you may be worried to hear that to use ports you have to use make. Don't be. The people who take care of the Ports infrastructure are the ones who really need to know how to deal with all the nuts and bolts of make. They've already solved all the common tasks so that the porters (those people who create the actual ports) can rely on it. This is done by including other Makefile fragments and it manages to hide away all the scariness. And for you as a user things are even simpler as you can just use what others created for you!
Let's take a look at the Makefile for our example port:
% cat /usr/ports/archivers/zstd/Makefile
# Created by: John Marino <marino@FreeBSD.org>
# $FreeBSD: head/archivers/zstd/Makefile 448492 2017-08-21 20:44:02Z sunpoet $
PORTNAME= zstd
PORTVERSION= 1.3.1
DISTVERSIONPREFIX= v
CATEGORIES= archivers
MAINTAINER= sunpoet@FreeBSD.org
COMMENT= Zstandard - Fast real-time compression algorithm
LICENSE= BSD3CLAUSE GPLv2
[...]
post-patch:
@${REINPLACE_CMD} -e 's|INSTALL_|BSD_&|' ${WRKSRC}/lib/Makefile ${WRKSRC}/programs/Makefile
.include <bsd.port.mk>
Now _that_ doesn't look half bad for a Makefile, does it? In fact it's mostly just defining variables! The only line that looks somewhat complex is the "post-patch" command (which is also less terrifying than it first looks - if you know _sed_ you can surely guess what it'll do).
There can actually be more files in some ports. If FreeBSD-specific patches are required to build the port, those are included in the ports tree. You can find them in a sub-directory called _files_ located in the port's directory. Here's an example:
% ls /usr/ports/editors/vim/files/
patch-src-auto-configure vietnamese_viscii.vim
patch-src-installml.sh vimrc
The patches there are named after the files that they apply to. Every patch in the files directory is automatically applied when building the port.
Alright. With that we've got a basic overview of what Ports are covered. The next post will show how to actually use them to build and install software.