Monitor your remote host network quality using smokeping on OpenBSD

Comment on Mastodon

Introduction

If you need to more the network quality of a link, or the network availability of a remote host, I'd recommend you to take a look at Smokeping.

Smokeping official Website

Smokeping is a Perl daemon that will regularly run a command (fping, some dns check, etc…) multiple times to check the availability of the remote host, but also the quality of the link, including the standard deviation of the response time.

It becomes very easy to know if a remote host is flaky, or if the link where Smokeping runs isn't stable any more when you see that all the remote hosts have connectivity issues.

Let me explain how to install and configure it on OpenBSD 7.2 and 7.3.

Installation

Smokeping comes in two parts, but they are in the same package, the daemon components to run it 24/7 to gather metrics, and the fcgi component used to render the website for visualizing data.

First step is to install the `smokeping` package.

# pkg_add smokeping

The package will also install the file `/usr/local/share/doc/pkg-readmes/smokeping` giving explanations for the setup. It contains a lot of instructions, from the setup to advanced configuration, but without many explanations if you are new to smokeping.

The daemon

Once you installed the package, the first step is to configure smokeping by editing the file `/etc/smokeping/config` as root.

Under the `*** General ***` section, you can change the variables `owner` and `contact`, this information is displayed on Smokeping HTML interface, so if you are in company and some colleague look at the graphs, they can find out who to reach if there is an issue with smokeping or with the links. This is not useful if you use it for yourself.

Under the `*** Alerts ***` section, you can configure the emails notifications by configuring `to` and `from` to match your email address, and a custom address for smokeping emails origin.

Then, under `*** Targets ***` section, you can configure each host to monitor. The syntax is unusual though.

That's for the simplest configuration file. It's possible to add new probes such as "SSH Ping", DNS, Telnet or LDAP...

Let me show a simple example of targets configuration I'm using:



probe = FPing

menu = Top
title = Network Latency Grapher
remark = Welcome to the SmokePing

+ Remote
menu= Remote
title= Remote hosts

++ Persopw

menu = perso.pw
title = My server perso.pw
host = perso.pw

++ openportspl

menu = openports.pl
title = openports.pl VM at openbsd.amsterdam
host = openports.pl

++ grifonfr

menu = grifon.fr
title = grifon.fr VPN endpoint
host = 89.234.186.37

+ LAN
menu = Lan
title = Lan network at home

++ solaredge

menu = solaredge
title = solardedge
host = 10.42.42.246

++ modem

menu = ispmodem
title = ispmodem
host = 192.168.1.254

Now you configured smokeping, you need to enable the service and run it.

# rcctl enable smokeping
# rcctl start smokeping

If everything is alright, `rcctl check smokeping` shouldn't fail, if so, you can read `/var/log/messages` to find why it's failing. Usually, it's a `+` line that isn't valid because of a non-authorized character or a space.

I recommend to always add a public host of a big platform that is known to be working reliably all the time, to have a comparison point against all your other hosts.

The Web Interface

Now the daemon is running, you certainly want to view the graphs produced by Smokeping. Reusing the example from the pkg-readme file, you can configure httpd web server with this:

    server "smokeping.example.org" {
	listen on * port 80
	location "/smokeping/smokeping.cgi*" {
	    fastcgi socket "/run/smokeping.sock"
	    root "/"
	}
    }

Your service will be available at the address `http://smokeping.example.org/smokeping/smokeping.cgi`.

For this to work, we need to run a separate FCGI server, fortunately packaged as an OpenBSD service.

# rcctl enable smokeping_fcgi
# rcctl start smokeping_fcgi

Note that there is a way to pre-render all the HTML interface by a cron job, but I don't recommend it as it will drain a lot of CPU for nothing, except if you have many users viewing the interface and that they don't need interactive zoom on the graphs.

Conclusion

Smokeping is very effective because of the way it renders data, you can easily spot issues in your network that a simple ping or response time wouldn't catch.

Please note it's better to have two smokeping setup at different places in order to monitor each other remote smokeping link quality. Otherwise, if a remote host appear flaky, you can't entirely be sure if the Internet access of the smokeping is flaky, or if it's the remote host, or a peering issue.

Here is the 10 days graph for a device I have on my LAN but connected to the network using power line networking.

Monitoring graph of a device connected on LAN using power line network

Don't forget to read `/usr/local/share/doc/pkg-readmes/smokeping` and the official documentation if you want a more complicated setup.