💾 Archived View for gemini.circumlunar.space › users › kraileth › neunix › 2021 › bsd_router_take_2_… captured on 2022-04-28 at 19:10:16. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2021-12-05)
-=-=-=-=-=-=-
Wednesday, 05. May 2021
[This article has been bi-posted to Gemini and the Web]
After OPNsense announced that they would rebase on vanilla FreeBSD instead of going on with HardenedBSD, I wrote the previous article on what "security" means when it comes to networked devices that are connected to the Internet. It also took a look at the fields where FreeBSD is doing pretty good. There's also the other side of the coin however. Being a person who really likes FreeBSD and enjoys working with it, this article is not one that I looked forward to writing. But FreeBSD is not all sunshine and roses (who would have thought?). And people should be aware of that to make an educated choice. So here we go.
It's the last two points from the previous article's list that FreeBSD admittedly sucks at. As discussed there, you _can_ make your FreeBSD systems a lot more secure than they are after a default installation. But FreeBSD does not believe in a "one size fits all" security concept. Truth be told, I'd also be _very_ skeptical about such a concept. Either it will be so general that it's not worth even laughing about or it will be highly theoretical and probably collide with real deployments pretty soon.
FreeBSD provides a lot of documentation to help you understand the system. However only you know your specific use case and therefore only you can put together the best possible security concept. I strongly support this way of thinking. It _is_ true however that FreeBSD could do a lot better with regards to basic security. Why isn't a firewall active by default? If it was that would probably be ipfw with a simplistic rule set. One of the first things that I'd do on a new installation would be disabling it and configuring pf instead. And if somebody _really_ has a use case where there should be no firewall - well, nobody would stop you from just disabling it. Coming with any firewall enabled by default would neither limit your choices nor would it make FreeBSD unfit for any sensible scenario. Making a change like that has POLA implications, though (FreeBSD's "Policy of least astonishment"). But if the will was there, a way would be found.
There's other things that make it really hard to come up with an excuse for. Frankly speaking: When it comes to mitigation techniques, FreeBSD is hardly a modern operating system. Linux, Windows - basically every common system you can name did a whole lot of work in that area over the last two decades. In comparison FreeBSD did almost _nothing_. Unfortunately there is not much more to say about that.
And one of the few things that were done, wasn't done _right_ according to some security researchers. FreeBSD claims to support ASLR (Address space layout randomization). It's not enabled by default, but it's there:
# sysctl -a | grep aslr
kern.elf32.aslr.stack_gap: 3
kern.elf32.aslr.honor_sbrk: 1
kern.elf32.aslr.pie_enable: 0
kern.elf32.aslr.enable: 0
kern.elf64.aslr.stack_gap: 3
kern.elf64.aslr.honor_sbrk: 1
kern.elf64.aslr.pie_enable: 0
kern.elf64.aslr.enable: 0
vm.aslr_restarts: 0
The idea of this mitigation technique is to randomly arrange data of processes in memory to make it harder for an attacker to hit a targeted function. In general ASLR's effectiveness as a mitigation has been doubted by a lot of people who are into security. Then again, it does not come with a high cost and so it's often seen as a baseline of protection. It's been standard in OpenBSD since 2003, in Linux since 2005 and in OS X as well as in Windows since 2007. NetBSD was a bit late to the party and only implemented it in 2009. Heck, even _Oracle Solaris_ adopted ASLR in 2012. And FreeBSD? Totally took their time. Got it first in 12.1 which was released in late 2019. But better late than never and maybe the wait was worth it if we got a superior implementation for that?
There's one problem, though: FreeBSD's implementation has been critizised as half-baked... According to Shawn Webb it's not even ASLR but ASR. So we got what we got extremely late, it's disabled by default and even if you enable it it's a pretty weak form of what is considered a very basic mitigation. This does not make FreeBSD look too good.
Let's poke another pain-point, shall we? FreeBSD is much, much less active in cleaning up their system than OpenBSD for example. If you take a look at the source repository, you won't have to search too long to find things that are not all that pretty. Here's one example:
FreeBSD source commit updating the comcontrol manpage
The commit removes a reference to the sio(4) interface in the comcontrol(8) manpage. This change is available in the recently released FreeBSD 13.0, older supported releases 12.2 and 11.4 still mention the interface. The thing is that sio(4) was removed from the GENERIC kernel in 2008 - which means that the manpage change that finally shipped this year (and for the very latest version only!) could easily have been shipped with FreeBSD 8.0 over ten years ago...
Want one more? Have a look:
FreeBSD source directory for pnpinfo
Meet "pnpinfo" which according to the manpage "reports information about _Plug-n-Play ISA devices_"! Hasn't been touched in over ten years and is very obviously completely obsolete. It's not built by default anymore, can't be built manually either (due to a missing system header file) - but it's still there in the source tree. It looks like it was still part of FreeBSD 10.0 (early 2014) but removed for 10.1 (late 2014). Till the end, the pnpinfo(8) manpage referenced pnp(4) which in turn had already been removed in FreeBSD 4.6 (2002)!
Right, this is nitpicking around very minor issues. Basically every project has dusty corners and when it is the size of FreeBSD, it would be close to a miracle to _not_ have any. Still it's only two easy to find examples out of many that show one thing: There's room for improvement. Plenty actually. But while *BSD prides itself in good documentation, little leftovers like this don't have such a huge impact on security. Could really be much worse right? For example if little to no maintenance was being done on very important system components responsible for, say secure authentication?
I can hear anything from deep sighs to screams of agony from readers familiar with FreeBSD even before I put a link to Heimdal in the base system here:
State of affairs in FreeBSD: Heimdal 1.5.2 in base
Really - FreeBSD ships with Heimdal 1.5.2 in the base system... This version was released in 2012 (!!) and nobody should have trouble believing me that there's a bunch of nasty CVEs for it. Right, everybody knows that you should _never_ use kerberos from the base system. If you need it, _always_ install either security/heimdal or security/krb5-$version from ports or packages. That way you'll get versions that are up to date. But honestly: Why _the heck_ is that ancient base system version even there? Nobody should have used it in almost a decade! What's the point in having a trap like that lurking in base? To see if unsuspecting users might fall into it and make an acquaintance of the poisoned spears at the bottom? That'll teach them a valuable lesson, eh? No, sorry. No point in even trying to whitewash this. It is just hideous and a real disgrace.
And then there's of course the recent turmoil around the flawed Wireguard implementation that almost made it into FreeBSD 13. If you don't know what I'm talking about, consider skimming over this article:
Ars Technica on FreeBSD Wireguard implementation
This is actually not a good article and I expected more of Jim Salter; he does a podcast called "2.5 admins" together with FreeBSD developer Allan Jude and they discussed the topic a couple of days before Salter wrote the linked article, forgetting some of the important things and concentrating on minor matters to have "a good story"... It will introduce you to the drama, however. Keep in mind that Jim pretends that the flawed code was "probably" only removed because the original Wireguard inventor intervened even though FreeBSD developers _were_ looking at the code and there already were people unhappy with it (whereas he denies that the even more recent happenings around Linux and the University of Minnesota showed that things in Linux world are also far from perfect).
Bottom line: There's all kinds of problems in FreeBSD. From small cosmetics to heavy-duty stuff. But FreeBSD is an Open Source project. If you think about contributing fixes (even for the very simple things): By all means do so! It's not that FreeBSD wants to be in the sorry state it is in regarding certain areas. The project is taking new contributions with open arms. You'd help make the world a little bit better for many people. And there's plenty of valuable skills to acquire if you choose to go down that road. Doc committers in FreeBSD are equal in their rights to ports and source committers, by the way. If you've got a bit of time for it and an interest in tech (you're reading articles like this not because you don't care at all, do you?) seriously consider it.
At this point the sunshine that the previous article may have shown is probably gone and there are some pretty dark clouds in the sky. Don't let the problems that I pointed out here scare you away. Remember that the above was written by a FreeBSD user - not a _former_ user. Everything added up, FreeBSD is a decent platform that's not worse than any other. In fact it has a lot of advantages that help accepting some of the disadvantages. Be aware of the ugly part, though. It might bite you otherwise.
But is this a god-given situation that we cannot do anything about? Is it _either_ the really nice features and sane structure of FreeBSD _or_ better mitigations but much less overall usefulness of OpenBSD (alternatively the better mitigations but the chaotic mess that is Linux these days)? Fortunately not: Enter HardenedBSD.
Have a look at this image to get an idea of what HardenedBSD is doing:
HardenedBSD feature comparison (PNG)
It's only four security features listed there that OpenBSD has but HardenedBSD doesn't. Of course the comparison is not complete, missing out a several good things in OpenBSD like e.g. pledge. However HardenedBSD also has a lot that go even further than what OpenBSD does.
And that's really, really impressive. Keep in mind that HardenedBSD is basically FreeBSD with a ton of security improvements to it: It has ZFS, jails and all the good stuff. It's a bit less convenient to use (e.g. you will have to understand additional tools like _secadm_ to toggle certain mitigation features on or off for specific programs). It offers you the means to make system administration a fair bit more cumbersome - while making life _terribly hard_ for attackers. If you are serious about security and accept that there is no free lunch, you're willing to endure the additional restrictions for a huge gain in hardening your system.
HardenedBSD is a hardened but not a hard fork of FreeBSD. It tracks upstream FreeBSD and merges new code from there. The project also aims to develop security features outside of FreeBSD but to ultimately give the changes back. This would be a huge gain for security-focused FreeBSD users. A very small project however has also very little chance of getting FreeBSD to accept proposed hardening techniques. For that reason HardenedBSD needs every bit of support it can get.
For some time, HardenedBSD also had LibreSSL in base instead of OpenSSL. They had to switch back for the simple reason that the team was to small to keep up with the work required for such an invasive change along with all the other security improvements. And now that OPNsense has announced to ditch HardenedBSD, it will lose some more badly needed support.
So is it a hopeless case? Well, not quite. OPNsense was definitely the most prominent user of HardenedBSD but certainly not the only one. There are people and companies using it. There is being research done with it; see e.g. this "bunker jails" article:
Bunker jails for protecting bank IT infrastructure
Co-founder Shawn Webb also managed to get a foundation started for it and even to attract an impressive amount of donations last year. I'd say that $13,000 instead of 11,000 they had aimed for is not bad at all! Especially if you compare it to the NetBSD foundation which only managed to get about 24,000 of their 50,000 goal even though they are a much older and bigger project.
I've been thinking about using HardenedBSD instead of FreeBSD when I build my next workstation. I'll probably also use it when I reinstall my server and see how that goes. Both will probably things to write about here on the Neunix Gemlog.
Decisions like this are always a tradeoff and I'm not under the impression that the OPNsense team made this one without carefully considering the matter. In short-term I think that tracking mainstream FreeBSD will definitely benefit OPNsense. Here's a couple of reasons:
Sounds good, right? If you're willing to sacrifice the additional hardening of HardenedBSD it sure does. And I think that most people would in fact prefer to go down that route.
IMHO OPNsense is hurting itself in the long run, though. The major reason for ditching HardenedBSD is that it is too much of a niche platform after all. With OPNsense leaving it, it will become even more niche. It is a _very_ important project to eventually take FreeBSD into the right direction. Let's not underestimate the gem that we have here! Trying to increase adoption would be what we should be doing, not decreasing it further.
But I don't want to challenge the decision that has been made, write a petition and bring unrest to the community. What OPNsense needs is to continue evolving for the better. One goal that aligns perfectly with the new strategy is getting rid of some more quirks that OPNsense inherited from pfSense and rather doing things like FreeBSD does. This would benefit everybody.
And who knows: Perhaps we'll see something like "HardenedSense" in the future? Not as a fork but as a community build for people who prefer to stick with a hardened system for their packet filter needs. I hope that this is food for thought for some readers. Maybe we can start a discussion over at the forums or so. If there's anybody interested in this, please let me know.
Following the announcement of OPNsense to part ways with HardenedBSD, some users over on Reddit proposed to rebase on OpenBSD instead. Let's consider this for a moment.
OpenBSD is generally regarded as a very, very secure operating system. It has a great lot of mitigations in place, a nice and clean codebase and a reasonable-sized community. That's certainly appealing. People also frequently mention that it has a much newer version of Pf which would be very much beneficial for a project like OPNsense.
There's a couple of reasons why this is not as good an idea as it seems, though. I actually like what the OpenBSD people are doing. No, truth be told, I _admire_ their _security first_ stance and the fact that they are willing to take it to the extreme anytime. But... Exactly this makes it the wrong choice for anything like OPNsense:
Let's also address the misconception of "newer Pf on OpenBSD": This is not true. Pf originated in OpenBSD when they dropped (due to licensing issues) IPF which they used before and replaced it with their own packet filter. Pf was later ported to FreeBSD (and NetBSD). After those ports happened, OpenBSD continued to improve Pf. One thing that they did was revising the syntax. FreeBSD did not sync their Pf with OpenBSD anymore - but for a good reason! They had improved their version of Pf to make it perform much better with multi-core CPUs. Contributing those changes back to OpenBSD was hopeless since OpenBSD was largely not SMP-capable at that time. For that reason Pf on OpenBSD and Pf on FreeBSD _diverged_, up to the point where merging newer changes from OpenBSD was simply not feasible anymore.
So it's not that OpenBSD has "newer PF" - it's more like both OpenBSD and FreeBSD have distinct versions of Pf that are actively developed but are quite different despite the common name. Rebasing OPNsense on OpenBSD would not give the users a much better Pf. In fact the major user-visible advantage of OpenBSD's version of Pf - i.e. the simpler syntax - would _not_ even be user-visible on OPNsense as people use the GUI to create their rules! It would on the contrary mean that code changes would be required so that the OPNsense application responsible for the rules would generate the rules in the new syntax expected by OpenBSD's Pf.
There would also be a lot of other things to change. OpenBSD's networking works quite a bit differently (e.g. the system's hostname goes into /etc/myname instead of into /etc/rc.conf as used in FreeBSD). The init system is slightly different. Packaging works _very_ differently (not using /usr/local for example and the package managers are simply worlds apart). And so on.
FreeBSD is a solid operating system that's doing well overall but is severely lacking in certain areas. HardenedBSD offers all the benefits of FreeBSD without a lot of the weaknesses and is an innovating force when it comes to strong security. OPNsense leaving HardenedBSD behind is a sensible choice considering OPNsense alone but a very unfortunate move for the FreeBSD ecosystem as a whole. OpenBSD is not the right base for OPNsense either.
If you care for FreeBSD and security, please support HardenedBSD. Let's keep it going strong - maybe there's the chance of having a community edition of 22.1 and onward that's still going to be based on HardenedBSD if there is enough interest.
