💾 Archived View for gemini.circumlunar.space › users › kraileth › neunix › eerie › 2017 › updating_f… captured on 2021-12-05 at 23:47:19. Gemini links have been rewritten to link to archived content
-=-=-=-=-=-=-
Here I'm republishing an old blog post of mine originally from February 2017. The article has been slightly improved.
In the first post of this mini series I wrote about legacy systems and installing FreeBSD 4.11. The second one shows how to configure the fresh system for remote access, bootstrap Pkgsrc, install Subversion to checkout FreeBSD code and update the system to the stable branch. And part three mainly deals with upgrading OpenSSH and the compilers. This post details some more updates until we reach the final state that's possible with such an old system (without resorting to extreme means).
Updating FreeBSD 4.11 (1/4) - Blast from the past
Updating FreeBSD 4.11 (2/4) - Digging up old graves
Updating FreeBSD 4.11 (3/4) - Neophyte's notorious necromancy
So far we've built some packages from 2013 and before. Using a current pkgsrc tree won't work - the various pkgsrc tools that our system has are too old. It might not be too big a step, but we can use a tree from the second half of 2014. Of course the newer SSH that we built before is not currently in the path so we need to create a temporary symlink before we can use CVS again:
# ln -s /usr/local/temp/bin/ssh /usr/local/pkgsrc/bin/ssh
# rehash
# cd /usr/pkgsrc
# cvs -danoncvs@anoncvs.netbsd.org:/cvsroot get -rpkgsrc-2014Q3 -P pkgsrc
# mv pkgsrc 14
# rm /usr/local/pkgsrc/bin/ssh
This is what the system looks like package-wise at the beginning of part 4 (PNG)
Most of pkgsrc's tools make use of NetBSD's compatibility library. Unfortunately the version that comes with the new pkgsrc tree won't build anymore on an OS as old as FreeBSD 4.11. Same thing for libfetch. But the newer tools will work with older versions of those libs, too! So let's prepare those two - libfetch need's some more love to build:
# cd /usr/pkgsrc/13/pkgtools/libnbcompat
# bmake
# cd /usr/pkgsrc/13/net/libfetch
# cp Makefile Makefile.bak
# sed '14i\\
CFLAGS= -Wno-error' Makefile.bak > Makefile
# bmake
As a next step we're going to do two updates. Yes, in theory we could use "bmake update”"to update packages. We will **not** do that. The reason is that we needed to abuse pkgsrc quite a bit so far by mixing package versions from various trees. Since "bmake update" is a destructive command (it will happily uninstall programs as well as packages depending on them!) this can lead to all sort of fun things like unresolvable dependencies and such.
If you like pain, go ahead. I've been there and I can confirm that it does work for some packages. For a lot of them actually. But in those cases where it doesn't, it tends to do so much damage that you're better off starting over than trying to fix things... That's why I’ll show you a safer method instead: Build a package and update via pkg_add! Also it really starts to show how old the system is that we're trying to build rather new packages on. More and more of them require some trickery to persuade them to build - but hey, we're doing a gross thing here, anyway. So there's no real reason to complain!
# cd /usr/pkgsrc/14/pkgtools/pkg_install
# bmake extract
# rm -r work/libnbcompat/*
# rm -r work/libfetch/*
# cp -R /usr/pkgsrc/13/pkgtools/libnbcompat/work/libnbcompat-20120702/* /usr/pkgsrc/14/pkgtools/pkg_install/work/libnbcompat/
# cp -R /usr/pkgsrc/13/net/libfetch/work/libfetch-2.34/* /usr/pkgsrc/14/pkgtools/pkg_install/work/libfetch/
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/14/packages/All/pkg_install-20130902nb1.tgz
# cd /usr/pkgsrc/14/pkgtools/bootstrap-mk-files
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/14/packages/All/bootstrap-mk-files-20140516.tgz
We made it so far, now let's make a daring move and just download the latest stable pkgsrc tree - released in January 2017:
# cd /usr/pkgsrc
# fetch http://cdn.netbsd.org/pub/pkgsrc/stable/pkgsrc-2016Q4.tar.bz2
# tar xvjf pkgsrc-2016Q4.tar.bz2
# rm pkgsrc-2016Q4.tar.bz2
# mv pkgsrc 16
Since mid 2014, pkgsrc makes use of a new package, _cwrappers_. During my test run I somehow managed to just get this package built. Despite taking notes, I have *no idea* what I did to just make it work! It must have been something that looked like a dead end (which is why I didn't include it in my notes) but somehow provided "getline"... I tried to get it working again for almost one whole Sunday but for the life of me couldn’' find out what I previously did... In the end I gave up and tried to find another solution. I found one, but while it is way more complex, it at least means that I got rid of that nasty blocker again:
# cd /usr/pkgsrc/16/pkgtools/cwrappers
# bmake extract
# rm -r work/libnbcompat/*
# cp -R /usr/pkgsrc/13/pkgtools/libnbcompat/work/libnbcompat-20120702/* /usr/pkgsrc/16/pkgtools/cwrappers/work/libnbcompat/
# cp work/cwrappers-20161125/mi_vector_hash.c work/cwrappers-20161125/mi_vector_hash.c.bak
# cp work/cwrappers-20161125/fixup-libtool.c work/cwrappers-20161125/fixup-libtool.c.bak
# sed 's/stdint.h/inttypes.h/' work/cwrappers-20161125/mi_vector_hash.c.bak > work/cwrappers-20161125/mi_vector_hash.c
# sed 's/stdint.h/inttypes.h/' work/cwrappers-20161125/fixup-libtool.c.bak > work/cwrappers-20161125/fixup-libtool.c
# cp /usr/pkgsrc/14/pkgtools/cwrappers/files/bin/getline.c /usr/pkgsrc/16/pkgtools/cwrappers/work/cwrappers-20161125/getline.c.bak
# sed 's/ssize_t/size_t/' work/cwrappers-20161125/getline.c.bak > work/cwrappers-20161125/getline.c
# cp work/cwrappers-20161125/common.h work/cwrappers-20161125/common.h.bak
# sed '107i\\
size_t getline(char **, size_t *, FILE *);' work/cwrappers-20161125/common.h.bak > work/cwrappers-20161125/common.h
# cp work/cwrappers-20161125/Makefile work/cwrappers-20161125/Makefile.bak
# sed '14i\\
LIB_SRCS+= getline.c' work/cwrappers-20161125/Makefile.bak > work/cwrappers-20161125/Makefile
# bmake install clean clean-depends
Phew! Fortunately the next few updates are straight forward:
# cd /usr/pkgsrc/16/pkgtools/bootstrap-mk-files
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/bootstrap-mk-files-20160908.tgz
# cd /usr/pkgsrc/16/devel/bmake
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/bmake-20150505.tgz
# cd /usr/pkgsrc/16/net/tnftp
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/tnftp-20151004nb1.tgz
Next is another one that requires some patching:
# cd /usr/pkgsrc/16/pkgtools/digest/
# bmake extract
# cp work/digest-20160304/sha3.h work/digest-20160304/sha3.h.bak
# cp work/digest-20160304/keccak.c work/digest-20160304/keccak.c.bak
# cp work/digest-20160304/keccak.h work/digest-20160304/keccak.h.bak
# cp work/digest-20160304/sha3.c work/digest-20160304/sha3.c.bak
# sed 's/stdint.h/inttypes.h/' work/digest-20160304/sha3.h.bak > work/digest-20160304/sha3.h
# sed 's/stdint.h/inttypes.h/' work/digest-20160304/keccak.c.bak > work/digest-20160304/keccak.c
# sed 's/stdint.h/inttypes.h/' work/digest-20160304/keccak.h.bak > work/digest-20160304/keccak.h
# sed 's/stdint.h/inttypes.h/' work/digest-20160304/sha3.c.bak > work/digest-20160304/sha3.c
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/digest-20160304.tgz
Let's update _gettext_ first as a lot of packages need that one; _xz_ is one of the packages that is linked against the old one and since libintl received a soname bump, it needs to be rebuilt as well. As we want to update it anyway that's not too bad. But there are other packages that we cannot update which depend on the old lib. So we'll have to create a symlink to satisfy their need, too:
# cd /usr/pkgsrc/16/devel/gettext-lib
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/gettext-lib-0.19.8.1.tgz
# cd /usr/pkgsrc/16/archivers/xz
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/xz-5.2.2.tgz
# ln -s /usr/local/pkgsrc/lib/libintl.so.9 /usr/local/pkgsrc/lib/libintl.so.7
# cd /usr/pkgsrc/16/devel/gettext-tools
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/gettext-tools-0.19.8.1.tgz
Next in line is some more typical build dependencies:
# cd /usr/pkgsrc/16/devel/libtool-base
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/libtool-base-2.4.2nb13.tgz
# cd /usr/pkgsrc/16/devel/m4
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/m4-1.4.17.tgz
# cd /usr/pkgsrc/16/devel/bison
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/bison-3.0.4nb3.tgz
Just a few more packages and we'll have updated most packages that can be updated (a few like zip and nbpatch can't):
# cd /usr/pkgsrc/16/shells/bash
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/bash-4.4.005.tgz
# cd /usr/pkgsrc/16/lang/perl5
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/perl-5.24.0.tgz
# cd /usr/pkgsrc/16/devel/autoconf
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/autoconf-2.69nb7.tgz
# cd /usr/pkgsrc/16/devel/gmake
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/gmake-4.1nb3.tgz
Before we can touch the compiler again, we first need to update the two math libraries (and create another symlink so we can go on compiling):
# cd /usr/pkgsrc/16/devel/gmp
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/gmp-6.1.2.tgz
# ln -s /usr/local/pkgsrc/lib/libgmp.so.13 /usr/local/pkgsrc/lib/libgmp.so.11
# cd /usr/pkgsrc/16/math/mpfr
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/mpfr-3.1.5.tgz
This unfortunately breaks the compiler. But we can still resort to the old GCC3 to build GCC4 once more, right? Right:
# cp /usr/pkgsrc/13/distfiles/gcc-4.4.7.tar.bz2 /usr/pkgsrc/16/distfiles
# cp /root/.cshrc /root/.cshrc.bak
# sed 's:pkgsrc/gcc44:temp/gcc34:' /root/.cshrc.bak > /root/.cshrc
# source /root/.cshrc
# cc -v
While we probably still don't need Object-C or Java we could in fact build GCC with them this time. Java requires Python2.7 installed but that can actually be built from the 2014 tree! The problem is that building Java requires more RAM than is available on 32 bit machines and will for that reason fail. However Java is deactivated by default for GCC 4.4 in the 2016 tree. So let's just get rid of our custom options, build the default package and set the correct path again:
# cp /usr/local/pkgsrc/etc/mk.conf /usr/local/pkgsrc/etc/mk.conf.bak
# sed '/PKG_OPTIONS.gcc44/d' /usr/local/pkgsrc/etc/mk.conf.bak > /usr/local/pkgsrc/etc/mk.conf
# cd /usr/pkgsrc/16/lang/gcc44
# bmake package clean clean-depends
# pkg_add -uu /usr/pkgsrc/16/packages/All/gcc44-4.4.7nb7.tgz
Almost everything updated! (PNG)
Now we only have to restore the correct path and then we have the GCC4 back (with a newer patch level):
# cp /root/.cshrc.bak /root/.cshrc
# source /root/.cshrc
# cc -v
gcc version 4.4.7 (GCC)
There's one more package to build that needs a bit of extra care: _Pkgconf_. It's a simpler replacement for the older _pkg-config_ but it won't work for us out of the box:
# cd /usr/pkgsrc/16/devel/pkgconf
# bmake extract
# cp work/pkgconf-1.0.1/libpkgconf/stdinc.h work/pkgconf-1.0.1/libpkgconf/stdinc.h.bak
# cp work/pkgconf-1.0.1/getopt_long.h work/pkgconf-1.0.1/getopt_long.h.bak
# sed 's/stdint.h/inttypes.h/' work/pkgconf-1.0.1/libpkgconf/stdinc.h.bak > work/pkgconf-1.0.1/libpkgconf/stdinc.h
# sed 's/stdint.h/inttypes.h/' work/pkgconf-1.0.1/getopt_long.h.bak > work/pkgconf-1.0.1/getopt_long.h
# bmake install clean clean-depends
Finally the time has come to do what I wanted to do in the first place, provide a recent version of OpenSSH! Of course it's also necessary to generate new host keys once more. And then, just to prove everything works when the machine boots, let's just restart the machine after adjusting the sshd path:
# cd /usr/pkgsrc/16/security/openssh
# bmake install clean clean-depends
# rehash
# ssh -V
# ssh-keygen -f /usr/local/pkgsrc/etc/ssh/ssh_host_rsa_key -N '' -t rsa
# ssh-keygen -f /usr/local/pkgsrc/etc/ssh/ssh_host_dsa_key -N '' -t dsa
# mkdir -p /usr/local/pkgsrc/run
# cp /etc/rc.conf /etc/rc.conf.bak
# sed 's:temp/sbin:pkgsrc/sbin:' /etc/rc.conf.bak > /etc/rc.conf
# shutdown -r now
Generating new host keys for OpenSSH (PNG)
Now we need to remove the vierelf entry in ~/.ssh/known_hosts once more before we connect again. Doing so in verbose mode even shows that the 4.11 box now has a newer version of OpenSSH installed that the FreeBSD 11 workstation that I use to connect to it! 🤣
FreeBSD 4.11 running a newer OpenSSH than my FreeBSD 11.0 workstation! (PNG)
FreeBSD 4.11 is really, really, _really_ old now. But you can get surprisingly far in running somewhat modern software on it - more recent software at least than I initially thought would be possible! And you? What was your bet? Would you have guessed that I'd make it up to the 2016Q4 pkgsrc tree and even install the latest version of OpenSSL and OpenSSH?
Here's a little summary of some important program updates:
Not too bad, eh? The notable exception here is binutils. Newer versions would probably be possible but there's a gap in pkgsrc - which stuck with 2.17 for a long time and then directly moved to 2.22 which no longer builds on FreeBSD 4.11. GCC 4.5.3 does build BTW but something goes sideways and the comparison of stage 2 and 3 fails for quite some files.
I've met my initial goal to provide a newer version of OpenSSH, surpassing all expectations that I had. There's room for more of course but that's not worth another post. I'm going to add _sudo_ and since Python 2.7 can be built it might even be possible to manage the 4.11 servers using _salt-ssh_ (the ordinary SaltStack doesn't work as it requires ZeroMQ which looks like it cannot be built)! We also have a recent version of _bash_ and can thus do some pretty nifty things with the right .bashrc.
This whole adventure took far longer than I had anticipated - a bit over a month instead of the intended two weekends! But that was mostly because I decided to start over with a clean system several times to ensure that everything works as I wrote it down here (and because GCC4 simply takes so long to build on the only spare machine that I had for this...). But it has been an interesting ride and I don't regret spending some time on the legendary FreeBSD 4.11!
Oh, and my special thanks to everybody involved with Pkgsrc! I usually don't have much use for NetBSD, but Pkgsrc is extremely useful. I might use it in the future on other systems (like Linux), too. And thanks to you for reading. I hope that you enjoyed it as well!
