FreeBSD vnet jails with jng

Posted on 2022-02-10

💬 Introduction

FreeBSD 13.0-RELEASE has a script in /usr/share/examples/jails/jng that can be used to quickly create/destroy vnet interfaces for jails using netgraph(4). It's a simple bridging setup where the jails appear as just another server on the network which is perfect for private networks.

jng netgraph topology for a simple jail

All the tools needed to create jails like this are in the base system.

✓ Preparations

With this setup assigning IP addresses to the jails needs to happen inside of them. I prefer to use DHCP for it which requires some initial setup. We to unhide bpf(4) by first setting a devfs ruleset in /etc/devfs.rules and later assigning it to the jails in /etc/jail.conf:

[devfsrules_jail_jng=11]
add include $devfsrules_jail_vnet
add path 'bpf*' unhide

Reload the rules:

$ service devfs restart

Load ng_ether(4):

$ sysrc kld_list+=ng_ether
$ service kld restart

Prepare /etc/jail.conf with some common variables for our vnet jails. em0 is the name of the external interface the jail interfaces are bridged to.

$jng = "sh /usr/share/examples/jails/jng";
$extif = "em0";
path = "/usr/jails/${name}";

host.hostname = "${name}.tobik.me";
vnet.interface = "ng0_${name}";
vnet = new;

exec.consolelog = "/var/log/jail_${name}_console.log";
exec.prestart = "$jng bridge $name $extif";
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown jail";
exec.release = "$jng shutdown $name";
exec.clean;

mount.devfs;
devfs_ruleset = 11;

Run jails on system startup:

$ service jail enable

⚠ If your FreeBSD host is running inside Hyper-V and if your jails need to reach the external network you might need to enable »Settings→Network Adapter→Advanced Features→Enable MAC address spoofing«.

🧪 Setting up jails

Create ZFS dataset for the jail:

$ zfs create zpool/usr/jails/vnetjail

Install FreeBSD inside the jail:

$ bsdinstall jail /usr/jails/vnetjail

Apply errata and security fixes to the jail:

$ freebsd-update -b /usr/jails/vnetjail fetch install

Enable DHCP in the jail. Normally devd(8) would start dhclient(8) asynchronously for interfaces marked with »DHCP«. However devd is not running inside jails (marked as nojail in /etc/rc.d/devd), so we need to enable either synchronous_dhclient=YES or mark the interface »SYNCDHCP« to get it to start as part of the normal jail boot.

$ sysrc -f /usr/jails/vnetjail/etc/rc.conf ifconfig_ng0_vnetjail=SYNCDHCP

Set it up in /etc/jail.conf:

$ echo 'vnetjail {}' >>/etc/jail.conf

Start it immediately:

$ service jail start vnetjail