Extending fail2ban on NixOS

Comment on Mastodon

Introduction

Fail2ban is a wonderful piece of software, it can analyze logs from daemons and ban them in the firewall. It's triggered by certain conditions like a single IP found in too many lines matching a pattern (such as a login failure) under a certain time.

What's even cooler is that writing new filters is super easy! In this text, I'll share how to write new filters for NixOS.

fail2ban GitHub project page

NixOS official website

Terminology

Before continuing, if you are not familiar with fail2ban, here are the few important keywords to understand:

For instance, a sshd jail will have a filter applied on sshd logs, and it will use a banning action. The jail can have more information like how many times an IP must be found by a filter before using the action.

Configuration

Enabling fail2ban

The easiest part is to enable fail2ban. Take the opportunity to declare IPs you don't want to block, and also block IPs on all ports if it's something you want.

  services.fail2ban = {
    enable = true;
    ignoreIP = [
      "192.168.1.0/24"
    ];
  };

  # needed to ban on IPv4 and IPv6 for all ports
  services.fail2ban = {
    extraPackages = [pkgs.ipset];
    banaction = "iptables-ipset-proto6-allports";
  };

Creating new filters

A filter is composed of one/many regex, and also a systemd journal unit in case you are pulling information from it instead of a log file.

We will use the module `environment.etc` to create files in `/etc/fail2ban/filter.d/` directory, so they can be used in the jails.

These are examples of filters you may want to use. They target very large, this may not be ideal for your use case, but can serve as a good start.

  environment.etc = {
    "fail2ban/filter.d/molly.conf".text = ''
      [Definition]
      failregex = <HOST>\s+(31|40|51|53).*$
    '';

    "fail2ban/filter.d/nginx-bruteforce.conf".text = ''
      [Definition]
      failregex = ^<HOST>.*GET.*(matrix/server|\.php|admin|wp\-).* HTTP/\d.\d\" 404.*$
    '';

    "fail2ban/filter.d/postfix-bruteforce.conf".text = ''
      [Definition]
      failregex = warning: [\w\.\-]+\[<HOST>\]: SASL LOGIN authentication failed.*$
      journalmatch = _SYSTEMD_UNIT=postfix.service
    '';
  };

Defining the jails using our new filters

Now we can declare fail2ban jails with each filter we created. If you use a log file, make sure to have `backend = auto`, otherwise the systemd journal is used and this won't work.

The most important settings are:

  services.fail2ban.jails = {

    # max 6 failures in 600 seconds
    "nginx-spam" = ''
      enabled  = true
      filter   = nginx-bruteforce
      logpath = /var/log/nginx/access.log
      backend = auto
      maxretry = 6
      findtime = 600
    '';

    # max 3 failures in 600 seconds
    "postfix-bruteforce" = ''
      enabled = true
      filter = postfix-bruteforce
      findtime = 600
      maxretry = 3
    '';

    # max 10 failures in 600 seconds
    "molly" = ''
      enabled = true
      filter = molly
      findtime = 600
      maxretry = 10
      logpath = /var/log/molly-brown/access.log
      backend = auto
    '';
  };

Creating filters

It's actually easy to create filters, fail2ban provides a good framework like automatic date and host detection, which make creating regex very easy.

You can use the command `fail2ban-regex` to experiment with regexes on some logs.

Here is an example of a log file that would contain an IP and an error message:

fail2ban-regex /var/log/someservice.log "<HOST> ERROR"

Here is an example of a systemd unit log that would contain an IP, then a space and a 403 error:

fail2ban-regex -m _SYSTEMD_UNIT=someservice.service systemd-journal "<HOST> 403"

You can analyze what lines matched or not with the flags `--print-all-matched` and `--print-all-missed`.

I recommend you to read fail2ban man pages and --help output if you want to create filters.

Conclusion

Fail2ban is a fantastic tool to easily create filtering rules to ban the bad actors. It turned out most rules didn't work out of the box, or were too narrow for my use case, so extending fail2ban was quite straightforward.