💾 Archived View for d.moonfire.us › blog › 2024 › 01 › 05 › teaching-nixos-about-opentofu captured on 2024-05-26 at 14:35:15. Gemini links have been rewritten to link to archived content

-=-=-=-=-=-=-

Teaching NixOS about OpenTofu

Up a Level

In my endless quest to come up with a completely data-driven and reproducible environment, I decided to take a stab at a new automation tool: OpenTofu[1]. I've already gotten a good NixOS[2] setup, but I wanted to also be able to check in the setup for my instances (and to a smaller degree, my bare metal servers in my home lab) to expand on the functionality. It didn't hurt that work had settled on Terraform.

1: https://opentofu.org/

2: /tags/nixos/

Previously, I had taken a stab at Pulumi[3] (during my wedding anniversary trip in 2022). It was fun and I liked the code but I ended up gutting it later. For some reason, it quickly ended up feeling like a chore to play with. At that point, I figured I would just do things manually. But then I saw the announcement that OpenTofu had forked from Terraform because of enshittification of licenses (develop with an open license, then switch to a more limited one once profits became important). That little thing set me off and I decided to try it out.

3: https://www.pulumi.com/

Installing

Since all my infrastructure code is in a Nix flake, to get started just required me to add to the shell's packages.

devShell.${system} = pkgs.mkShell {
  buildInputs = [
    pkgs.just
    pkgs.opentofu
    pkgs.openstackclient
  ];
};

I also grabbed the OpenStack client because it made easier to find some of the nasty little identifiers I needed to import.

Configuration Files

The way tofu works, it grabs all the `*.tf` files in the same directory. So inside my infrastructure flake, I have a `src/tofu` directory with configuration files that make sense to me:

`000-providers.tf`
`050-secrets.tf` (.gitignored)
`050-secrets.tf.enc` (SOPS encrypted based on my user key)
`200-networks.tf`
`500-instance0.tf`
`500-instance1.tf`

All of them are picked up, merged together, and made into a single set of settings. I also use `tofu fmt` a lot since I like to normalize my files on every commit.

OpenStack and DreamHost

Fortunately, my hosting provider of choice is DreamHost[4]. They aren't the cheapest or the best, but they appear to be ethetical. Mostly, I stick with them because they went to the court to fight some overreaching gag orders[5].

4: https://dreamhost.com/

5: https://techfreedom.org/victory-online-political-free-speech-dreamhost-case/

(I also tried DigitalOcean at the same time as Pulumi but dropped that also.)

OpenTofu (via Terraform plugins) does a wonderful job of supporting both OpenStack and DreamHost DNS to tie everything together.

terraform {
  required_version = ">= 0.14.0"

  required_providers {
    dreamhost = {
      source  = "adamantal/dreamhost"
      version = "0.3.2"
    }

    openstack = {
      source  = "terraform-provider-openstack/openstack"
      version = "~> 1.53.0"
    }
  }
}

I added the `adamantal/dreamhost` plugin so I could also assign the DNS record directly from my script and have everything working.

Secrets

Even though my infrastructure flake is in a private repository, I still encrypt all my secrets. I use SOPS for this, which means setting up the `.sops.yaml` file to encrypt and then decrypting/encrypting files using `just``:

decrypt: decrypt-clouds decrypt-secrets

decrypt-clouds:
    if [ ! -f clouds.yaml ];then sops -d clouds.yaml.enc > clouds.yaml;fi

encrypt-clouds:
    cp clouds.yaml clouds.yaml.enc
    sops -i -e clouds.yaml.enc

decrypt-secrets:
    if [ ! -f 050-secrets.tf ];then sops -d 050-secrets.tf.enc > 050-secrets.tf;fi

encrypt-secrets:
    cp 050-secrets.tf 050-secrets.tf.enc
    sops -i -e 050-secrets.tf.enc

I do the same with my `.env` file so I can get the information I need set up properly.

Creating an Instance

Here is a short segment for creating an instance on DreamHost (or most OpenStack providers).

resource "openstack_compute_instance_v2" "instance1" {
  provider    = openstack.dreamhost
  name        = "instance1" // It really isn't instance1, but just pretend
  key_pair    = "keypair1" // This is my SSH key set up somewhere else
  flavor_name = "gp1.supersonic" // gp1.supersonic means I don't need a swap disk

  user_data = <<-EOT
    #cloud-config
    runcmd:
      - curl https://raw.githubusercontent.com/elitak/nixos-infect/master/nixos-infect | NIX_CHANNEL=nixos-unstable bash 2>&1 | tee /tmp/infect.log
    EOT

  # This sets up the boot device as /
  block_device {
    source_type           = "image"
    uuid                  = "2b2c61c6-324c-47f4-88c1-9ae8a978ddfd" # Ubuntu
    boot_index            = 0
    delete_on_termination = true
    destination_type      = "volume"
    multiattach           = false
    volume_size           = 80
  }

  network { // Also configured somewhere else
    name = openstack_networking_network_v2.public.name
  }
}

resource "dreamhost_dns_record" "instance1" {
  record = "instance1.mfgames.com"
  value  = openstack_compute_instance_v2.instance1.network[0].fixed_ip_v4
  type   = "A"
}

For me, the part is really cool is that I can bake in the NixOS infect[6] script right in. To my surprise, it just ran the first time without errors (thought I had to wait about ten minutes after OpenTofu said it was done).

6: https://github.com/elitak/nixos-infect

All I had to do was either show the results:

tofu plan

Or apply the changes:

tofu apply

Importing an Instance

Actually, the first thing I did was import my existing instances into the system. This involves creating a `.tf` file with the same basic setup at the other instance (some fields can be skipped but I was still learning), then import with the ID from OpenStack. Of course, getting the IDs was the hard part. Fortunately, this is where the OpenStack client comes into play. I can use that to get the list of servers, figure out the ID, then import it into Tofu.

$ openstack --os-cloud dreamhost server list
+--------------------------------------+-----------+--------+------------+--------------------------+----------------+
| ID                                   | Name      | Status | Networks   | Image                    | Flavor         |
+--------------------------------------+-----------+--------+------------+--------------------------+----------------+
| 55f8ee35-31b2-4137-af1d-b7597d348271 | instance0 | ACTIVE | public=*** | N/A (booted from volume) | gp1.supersonic |
| 1a38092b-bbc5-46bd-9092-0df979ca8fe4 | instance1 | ACTIVE | public=*** | N/A (booted from volume) | gp1.supersonic |
$ tofu import openstack_compute_instance_v2.instance0 55f8ee35-31b2-4137-af1d-b7597d348271

NixOS

Now, while this was great for setting up things, I also wanted to pull that data into my Nix infrastructure flake. Fortunately, OpenTofu has a way of exporting the data pulled from the cloud. To do that, I need to add an `output` stanza at the bottom of my `500-instance1.tf` file:

output "instance1_ipv4" {
  value = openstack_compute_instance_v2.instance1.network[0].fixed_ip_v4
}

Since I'm (recently) fond of using Just[7] for automation, I banged up a little stanza that automatically creates a `default.nix` file inside that directory every time I apply.

7: /tags/just/

apply: decrypt format && export
    tofu apply

format:
    tofu fmt

plan: decrypt format
    tofu plan

export:
    echo "inputs: {" > default.nix
    tofu output | sort | perl -ne 'chomp;s@_@.@g;print "  $_;\n"' >> default.nix
    echo "}" >> default.nix

I had to use `_` in the output since dotted notation isn't accepted, but I use `perl` to convert those underscores into Nix-happy format.

inputs: {
  instance0.ipv4 = "1.2.3.4";
  instance1.ipv4 = "2.3.4.5";
}

I use this to pull into my `networking.nix` which is used to drive things like configuring AdGuard, services like Maddy (for DeltaChat) and other services.

inputs:
let
  tofu = import ./tofu/default.nix { };
in
{
  instance0 = tofu.instance0;
  instance1 = tofu.instance1;
  instance2.ipv4 = "192.168.0.2";
}

From there, I have a single place to get all my IP addresses:

inputs:
let
  ip = (import ../../../../networks.nix { }).instance0.ipv4;
in
{
}

Conclusion

It isn't the best or most graceful way of doing things, but I'm pretty happy how everything turned out. I made a few mistakes along the way of setting up Gitea Actions and had to drop and rebuild my instance0. That was just a matter of renaming `500-instance0.tf`, applying to drop, and then name the file back. Then I had nice clean slate to push out a new closure.

OpenTofu is much nicer than Pulmui. It didn't insist on having a cloud to maintain state, the file is checked into Git instead. It has a declarative language instead of code, and since I really don't need a lot of that logical flow, it just works for me. Plus I was able to inject into my `just deploy` top-level script that pushes out changes to my home lab and all my instances in a single call.