💾 Archived View for outsidethe.net › articles › nixgemini.gmi captured on 2023-06-16 at 15:56:17. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2023-01-29)
-=-=-=-=-=-=-
This article details how I use Nix[1] to build a container image that can publish the same Markdown documents over both HTML and Gemini[2].
The layout of the directory structure to do this is below. We'll discuss each of these files as we go.
.
├── bin
│ ├── debug
│ └── deploy
├── content
│ └── public
│ ├── articles
│ │ └── nixgemini.md
│ ├── index.md
│ └── markdown.html
├── flake.lock
├── flake.nix
├── fly.toml
├── result
└── static
├── Caddyfile
├── Caddyfile.dev
└── Caddyfile.main
The Caddy[3] webserver has the ability to transform Markdown into HTML natively. Two files achieve this, Caddyfile and Caddyfile.main.
Caddyfile:
outsidethe.net import Caddyfile.main
Caddyfile.main:
root * /public
file_server
templates
encode gzip zstd
try_files {path} {path}.html
rewrite * /markdown.html
The reason for the seemingly pointless import is that Caddyfile.dev contains:
localhost:80 import Caddyfile.main
which allows us to start Caddy without any complications from the automatic TLS certificate generation. Later on we'll see how we produce a debugging container that uses this.
The rewrite configuration option takes every path and renders /markdown.html regardless. It is in here we use Caddy's Go templating to render the markdown at the correct path. Let's look at the contents:
[[$pathParts := splitList "/" .OriginalReq.URL.Path]]
[[$markdownFilename := default "index" (slice $pathParts 2 | join "/")]]
[[$markdownFilePath := "" ]]
[[if eq $markdownFilename "index"]]
[[$markdownFilePath = printf "/%s.md" $markdownFilename]]
[[else]]
[[$markdownFilePath = printf "/articles/%s.md" $markdownFilename]]
[[end]]
[[if not (fileExists $markdownFilePath)]][[httpError 404]][[end]]
[[$markdownFile := (include $markdownFilePath | splitFrontMatter)]]
[[$title := default $markdownFilename $markdownFile.Meta.title]]
<!DOCTYPE html>
<html>
<head>
<title>outsidethe.net</title>
</head>
<body>
[[markdown $markdownFile.Body]]
</body>
</html>
(I've substituted double braces for square braces above to avoid complications with double-rendering of templates).
The upshot of all of this is that if there is a Markdown file at the correct path it is rendered as HTML.
For the Gemini versions of each Markdown we generate a .gmi file using the md2gemini program. This is done as part of preparing the contents of the container image and I'll talk about that in a bit more detail shortly.
The Gemini files are served with the Agate server[4] which is very similar to Caddy insofar as it also handles TLS certificates for us. One downside to the manner in which I deploy all of this is that each new deployment generates a new TLS certificate. Since Gemini works on the basis of trust-on-first-use this means that frequent deployments could be bothersome for those who visit often. That said, being Gemini, I don't think that's a great concern and certainly not one worth the extra complication of persistent volumes.
We bring all of this together in a Nix Flake that looks like this:
{
description = "outsidethe.net";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
flake-utils.url = "github:numtide/flake-utils";
};
outputs = { self, nixpkgs, flake-utils }:
flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs { system = system; };
x86Linux = import nixpkgs { system = "x86_64-linux"; };
content = pkgs.stdenv.mkDerivation {
name = "content";
buildInputs = [ pkgs.md2gemini ];
src = ./content;
builder = builtins.toFile "builder.sh" ''
source $stdenv/setup
mkdir $out
cp -r $src/* $out
# The "source code" is immutable by default but
# we need to set the write bit on the directories
# to be able to generate new files alongside it.
chmod u+w $out/public
chmod u+w $out/public/articles
# Build the index page with the copy link style.
cd $out/public
md2gemini --links copy --plain --write index.md
# Fix article links to have .gmi extension
# (this presumes GNU sed).
sed -i "s|\(=> /articles/[a-z]\+\)|\1.gmi|g" index.gmi
# Build the articles with an alternative link style.
cd $out/public/articles
for f in $(find . -name "*.md"); do
md2gemini --links 'at-end' --plain --write $(basename $f)
done
'';
};
container = pkgs.dockerTools.buildImage {
name = "registry.fly.io/outsidethenet";
tag = "latest";
config = {
Cmd = [
"bash"
"-c"
"${x86Linux.caddy}/bin/caddy start; ${x86Linux.agate}/bin/agate --content /public --hostname outsidethe.net"
];
};
copyToRoot = [
content
./static
x86Linux.agate
x86Linux.bash
x86Linux.cacert
x86Linux.caddy
];
};
debugContainer = pkgs.dockerTools.buildImage {
name = "debug";
tag = "latest";
fromImage = container;
config = {
Cmd = [
"bash"
];
};
copyToRoot = [
x86Linux.bash
x86Linux.coreutils
x86Linux.curl
x86Linux.vim
];
};
in {
devShells.default = pkgs.mkShell {
nativeBuildInputs = [
pkgs.agate
pkgs.amfora # Browser
pkgs.caddy
pkgs.castor # Browser
pkgs.flyctl
pkgs.lagrange # Browser
];
};
packages.container = container;
packages.content = content;
packages.debug = debugContainer;
packages.default = container;
}
);
}
Unusually, perhaps, we import nixpkgs twice:
pkgs = import nixpkgs { system = system; };
x86Linux = import nixpkgs { system = "x86_64-linux"; };
This is to ensure that no matter the host operating system we have the native standard library available (via pkgs) but can always build an x86_64-linux container image.
The first derivation we build contains the content we want to serve. This is the /content tree containing our Markdown and HTML. The custom builder for this derivation runs the md2gemini code and generates our .gmi files alongside their .md counterparts.
Apart from us having to do a slightly distasteful sed to ensure that the links (which will work without a file extension when served via HTML thanks to the try_files {path} {path}.html stanza in the Caddy configuration) are rewritten with a .gmi extension, which the Gemini specification insists upon it is all pretty straightforward.
Here I use the dockerTools.buildImage builder to ensure all of our dependencies are present within the image: the content, the configuration, the two pieces of server software and our root certificate chains. Notably, thanks to us constructing the content as a derivation we don't ship md2gemini in the deployed container image.
The debugContainer derivation produces another container that is byte-for-byte identical to the deployed container image but contains some additional software to make debugging any issues a little easier. I build and run this using the bin/debug script which looks like this:
#!/usr/bin/env bash nix build .#debug && \ docker load < result && \ docker run -it --rm -p 127.0.0.1:8080:80 -p 127.0.0.1:1965:1965 debug
I intentionally do not start any of the servers. If I wish to I can start Caddy with:
caddy run --config Caddyfile.dev
and, as mentioned before, avoid any complications with TLS certificate generation.
The final thing we configure in the Nix Flake is a development shell that contains the same software that we bake into the container images and a few different Gemini browsers to make it easy to see how things render.
Since this container image is hosted on fly.io[5] deployment is a simple matter of bin/deploy which contains:
#!/usr/bin/env bash nix build &&\ docker load < result &&\ docker push registry.fly.io/outsidethenet:latest &&\ fly deploy
The fly.toml configuration is as simple as I can make it:
app = "outsidethenet"
kill_signal = "SIGINT"
kill_timeout = 5
processes = []
[env]
[build]
image = "registry.fly.io/outsidethenet:latest"
# Gemini
[[services]]
internal_port = 1965
protocol = "tcp"
[services.concurrency]
hard_limit = 25
soft_limit = 20
type = "connections"
[[services.ports]]
port = 1965
[[services.tcp_checks]]
grace_period = "1s"
interval = "15s"
restart_limit = 0
timeout = "2s"
# HTTP
[[services]]
internal_port = 80
protocol = "tcp"
[services.concurrency]
hard_limit = 25
soft_limit = 20
type = "connections"
[[services.ports]]
port = 80
[[services.tcp_checks]]
grace_period = "1s"
interval = "15s"
restart_limit = 0
timeout = "2s"
# HTTPS
[[services]]
internal_port = 443
protocol = "tcp"
[services.concurrency]
hard_limit = 25
soft_limit = 20
type = "connections"
[[services.ports]]
port = 443
[[services.tcp_checks]]
grace_period = "1s"
interval = "15s"
restart_limit = 0
timeout = "2s"
2: https://gemini.circumlunar.space/