📅 Published 2021-08-22
If all goes well, you should be seeing a blog post right now. But how did it get here? Let's talk about tools, and the fun challenge of building a multi-protocol site on both Gemini and the legacy web.
If you're interested in publishing a similar site or converting an existing Hugo website to Gemini, I hope this helps you get started! Before we begin, the source code for this site is available on Sourcehut under an MIT license, so if you want to use it as a basis for your project then you're more than welcome to do so.
I've been thinking about putting up a blog for a long time, but I could never decide on a platform. The last thing I need is another server to manage, and I've never liked overly complicated blogging software. All I need is some Markdown and a simple static site to get my point across.
When Sourcehut added Gemini to their Sourcehut Pages feature then my interest in blogging was suddenly rekindled. Pages is similar to the offerings from GitHub and GitLab, but just a bit simpler. And I'm excited about the Gemini hosting! I've been watching Gemini eagerly for some time, as I used to enjoy browsing Gopher sites back when I was a kid, before the web took off. Gemini's simplicity reminds me of all the best parts of Gopher, without the cruft often found in antiquated protocols. Beyond this, I appreciate Sourcehut's minimalism, and I want to support small businesses within the free software ecosystem.
Manual publishing is possible, but I prefer to automate my builds. Git repositories on Sourcehut can use builds.sr.ht **(note: paid feature)** to automatically build and deploy a static site to Pages upon commit. You can use pretty much any static site builder to accomplish this, so I went with Hugo. I have no particular reason for this choice, other than a growing interest in Go and the novelty factor of using a new tool, but it turned out to be a good decision.
I won't be explaining much about my Hugo configuration in this post, as many other people have made excellent Hugo tutorials. One of the best ways to learn is by taking an existing site (such as this one) and changing small things about it. Do that and peruse the Hugo documentation and you'll be set.
After deciding on a platform, I had a new problem. How can I write my content once and deploy it to both the legacy web and Gemini? (Keep in mind that Gemtext looks like Markdown, but it's not 100% compatible.) I did some research, and I had a few obvious options:
This last option was preferable, especially if it could be done with little effort.
After a quick search, I came across gmnhg, a tool purpose-built for rendering Hugo content as a Gemini site. This seemed like the best option.
As it turns out, gmnhg is more than just a simple Hugo-to-Gemini conversion tool. It's really a full-fledged static site generator, with the ability to present Hugo content custom-tailored to fit a Gemtext format. This means that you must do some abbreviated templating, but this work is trivial compared to building Hugo templates, and the defaults are perfectly acceptable.
Here's an example of how inline links (not possible in Gemtext) are translated from Markdown. First, the original Markdown:
The [example.com](https://example.com) domain is a reserved domain used for examples in documentation. The list of reserved domains is managed by [IANA](https://www.iana.org).
Now the resulting Gemtext:
The example.com domain is a reserved domain used for examples in documentation. The list of reserved domains is managed by IANA. => https://example.com example.com => https://www.iana.org/ IANA
I've set up a WWW example page and a Gemini example page to demonstrate how elements in Markdown are translated into Gemtext by gmnhg. You may notice a few bugs and quirks; there are still a few gaps in what gmnhg can provide. I was able to solve many issues by submitting patches, and the author is responsive to bug reports and contributions. At this point it serves my needs.
Using the source code for blog.tdem.in (gmnhg author's blog) as a reference, I set up a basic project structure so I would have something to build on moving forward. This was a simple process. I may do a longer post on it some time in the future, as I've made some changes to my own templates that others may find useful.
Now I had to think about build enviroments. I don't use Hugo on a day-to-day basis, so I didn't want it or any related programs cluttering up my `$PATH`. I also had to consider automated builds on Sourcehut; gmnhg isn't officially packaged anywhere as of today, so I was going to have to trigger a fetch and build of it somehow. I also wanted to get all of this set up early on, so I could get a rapid code -> test -> commit -> deploy loop set up. It's always motivating to see your project deployed in a real environment, even if it's not really done yet!
For me this choice was easy. There's a million and one ways to manage development environments and remote builds, but I decided on Nix.
For the uninitiated, Nix is a tool originally designed to build packages in a reproducible way based on a functional definition. The scope of the project has... expanded somewhat. I came to Nix through NixOS, which extends Nix to allow for building and managing entire operating systems through a defined set of configurations.
Nix is an incredibly powerful tool with a reputation for being complex and difficult to learn, but I feel that this reputation is undeserved. So many build systems are completely arcane and opaque, whereas Nix is refreshingly simple as long as you have a basic understanding of functional programming. (And I mean *basic*. No monads required. If you've hand-edited a `.emacs` file, you've got this.)
I found NixOS easy enough to learn by following published examples and making small changes, but I hadn't yet spent any time with Nix as a development tool. Time to learn something new!
Nix provides a tool called `nix-shell` that was originally intended for debugging builds. As with everything Nix, the scope has grown, and `nix-shell` is now often used to provide virtual environments for development. The tool is configured by defining your configuration in either `default.nix` or `shell.nix`, files which should be placed in your project root.
Unfortunately, the official documentation for this use case is not all that helpful--perhaps because this use wasn't the original purpose of the tool--but I quickly found an excellent article titled "Manage a static website with Hugo and Nix" that explains how to create a `nix-shell` setup for Hugo blogs. This was more than enough to get me started.
Manage a static website with Hugo and Nix
For reference, here's the base `default.nix` from the article:
let
# See https://nixos.wiki/wiki/FAQ/Pinning_Nixpkgs for more information on pinning
nixpkgs = builtins.fetchTarball {
# Descriptive name to make the store path easier to identify
name = "nixpkgs-unstable-2019-02-26";
# Commit hash for nixos-unstable as of 2019-02-26
url = https://github.com/NixOS/nixpkgs/archive/2e23d727d640f0a96b167d105157f6e7183d8f82.tar.gz;
# Hash obtained using `nix-prefetch-url --unpack [url]`
sha256 = "15s7qjw4qm8mbimiv5fcg0nlgpx4gsws2kbx8z1qzqrid8jg76f8";
};
in
{ pkgs ? import nixpkgs {} }:
with pkgs;
let
hugo-theme-terminal = runCommand "hugo-theme-terminal" {
pinned = builtins.fetchTarball {
# Descriptive name to make the store path easier to identify
name = "hugo-theme-terminal-2019-02-25";
# Commit hash for hugo-theme-terminal as of 2019-02-25
url = https://github.com/panr/hugo-theme-terminal/archive/487876daf1ebdf389f03a2dfdf6923cea5258e6e.tar.gz;
# Hash obtained using `nix-prefetch-url --unpack [url]`
sha256 = "17gvqml1wl14gc0szk1kjxi0ya995bmpqqfcwn9jgqf3gdx316av";
};
patches = [];
preferLocalBuild = true;
}
''
cp -r $pinned $out
chmod -R u+w $out
for p in $patches; do
echo "Applying patch $p"
patch -d $out -p1 < "$p"
done
'';
in
mkShell {
buildInputs = [
hugo
];
shellHook = ''
mkdir -p themes
ln -snf "${hugo-theme-terminal}" themes/hugo-theme-terminal
'';
}
The linked article explains this in more detail, so I won't go into depth on it here. In short, with the above `default.nix` you get a few things:
My own configuration has diverged from the one provided in the article, as I will explain below. (Note: you can find my current default.nix configuration on Sourcehut.)
The first task is easy enough: updating the `nixpkgs` reference to a current version.
If you look at the configuration, you'll notice that `nixpkgs` has a url that points to an archive file on GitHub. This file contains the build definitions for all Nix packages under a specific Nix release.
The long alphanumeric identifier in this URL refers to a specific Git commit ID. In order to update your package definitions, you can visit the nixpkgs release page on GitHub and grab the hash for the version you want to use, then update the URL string. (For instance, nixpkgs 21.05 is `https://github.com/NixOS/nixpkgs/archive/7e9b0dff974c89e070da1ad85713ff3c20b0ca97.tar.gz`.) You should also update the name to provide an accurate description.
But that's not quite enough. If you run `nix-shell` now, you'll get an error. That's because Nix is watching out for you--it sees that the contents of the archive don't match the expected SHA256 sum. So first you must run `nix-prefetch-url --unpack [url]` to get the updated SHA256 hash, then copy that hash into your `default.nix` configuration.
For nixpkgs 21.05, you would run `nix-prefetch-url --unpack https://github.com/NixOS/nixpkgs/archive/7e9b0dff974c89e070da1ad85713ff3c20b0ca97.tar.gz` and you will receive the hash `1ckzhh24mgz6jd1xhfgx0i9mijk6xjqxwsshnvq789xsavrmsc36`. Replace the `sha256` value in `default.nix` with this hash. If you did everything right, you can run `nix-shell` again and you will successfully enter a shell after downloading and building any dependencies.
Here's `nixpkgs` updated to use the 21.05 release:
nixpkgs = builtins.fetchTarball {
# Descriptive name to make the store path easier to identify
name = "nixpkgs-21.05";
# Commit hash for nixpkgs release
url = https://github.com/NixOS/nixpkgs/archive/7e9b0dff974c89e070da1ad85713ff3c20b0ca97.tar.gz;
# Hash obtained using `nix-prefetch-url --unpack [url]`
sha256 = "1ckzhh24mgz6jd1xhfgx0i9mijk6xjqxwsshnvq789xsavrmsc36";
};
Updates like this will become easier with the upcoming Nix Flakes feature, but for now this process isn't too cumbersome. I don't intend to update my dependencies often once I achieve a stable configuration.
Now for something more difficult. In the article, the author used `runCommand` with `fetchTarball` to fetch a theme snapshot from GitHub and store it as a Nix package. We could do something similar for gmnhg, but there is a better solution in recent versions of Nix: `buildGoModule`.
The `buildGoModule` function greatly simplifies the process of building Go packages. Background information can be found in the article "Announcing the new Golang infrastructure: buildGoModule," but do note that the implementation has changed slightly since that article was published.
Announcing the new Golang infrastructure: buildGoModule
At time of writing, my configuration for building gmnhg looks like the following:
gmnhg = buildGoModule {
pname = "gmnhg";
version = "0.2.0-8cdbc77";
src = builtins.fetchTarball {
# Descriptive name to make the store path easier to identify
name = "hugo-gmnhg-0.2.0-8cdbc77";
# Gzip of commit hash
url = https://github.com/tdemin/gmnhg/archive/8cdbc778e53914ad7ac28f981590f1d8c7083b1a.tar.gz;
# Hash obtained using `nix-prefetch-url --unpack [url]`
sha256 = "0p3gw965hgpmhfw480mr9yh314mp05c5z7cfz4kki882w4pmbpjr";
};
# Get updated hash by setting to lib.fakeSha256
vendorSha256 = sha256:1j16j5sl45k7nf8zrrzcv5b7fvmqyp4v6hpmssf1j8449gda2l8v;
meta = with lib; {
description = "Hugo-to-Gemini Markdown converter";
homepage = "https://github.com/tdemin/gmnhg";
license = licenses.gpl3;
maintainers = with maintainers; [ tdemin ];
platforms = platforms.linux;
};
# Add path to patches below
# Example line: ./patches/gmnhg/my-patch.diff
patches = [];
};
As you can see, most of these are self-explanatory, with the exception of `fetchTarball` (which we've already discussed) and `vendorSha256`, which verifies the SHA256 sum of vendored dependencies. You can get an updated SHA256 hash by running `nix-shell`. You will get an error message and a newly updated hash, which you can copy into your `default.nix` configuration to fix the error. (If you are starting fresh and don't have any hash yet, you can set this value to `lib.fakeSha256` to accomplish the same thing.)
Since I am building for Gemini, I will need a way to test Gemini sites. I found that agate, a simple Gemini server, and gmni, a simple client, were both available in nixpkgs. I added these to my `buildInputs` along with the newly built gmnhg package.
buildInputs = [ hugo gmnhg agate gmni ];
This tells Nix to fetch these packages if needed and make them available in the `nix-shell` environment.
At this point I can build my site by first using `nix-shell` to enter the build environment, then following this with either `hugo build` to build the Hugo site or `gmnhg` to build the Gemini site.
Finally I am ready to set up automated builds. On Sourcehut, builds are started automatically upon commit if you have a build manifest defined in `.build.yml` under the root directory for your Git repository. This is documented thoroughly in the Sourcehut Pages tutorial and the official builds.sr.ht docs.
I was pleased to see that Sourcehut supports NixOS in its list of build images. This means that I get Nix support fresh out of the box, without installing anything.
I decided to build my Hugo site in `public/hugo` and my Gemini site in `public/gemini`. Following the example from the Sourcehut docs, I then gzip the output and upload it using `acurl`. A simple configuration for this looks like the following:
image: nixos/latest
oauth: pages.sr.ht/PAGES:RW
environment:
site: mntn.xyz
tasks:
- package: |
main=$PWD
cd $main/$site
nix-shell --run "hugo -d public/hugo --minify"
cd public/hugo
tar -cvz . > $main/site.tar.gz
cd $main/$site
nix-shell --run "gmnhg -output public/gemini"
cd public/gemini
tar -cvz . > $main/gemini.tar.gz
- upload: |
acurl -f https://pages.sr.ht/publish/$site -Fcontent=@site.tar.gz
acurl -f https://pages.sr.ht/publish/$site -Fprotocol=GEMINI -Fcontent=@gemini.tar.gz
For information on `acurl` and the general structure of this configuration file, see the Sourcehut docs.
Calling `nix-shell --run` here tells Nix to run the command specified, within the Nix shell environment described by `default.nix`. Before running the command, Nix will download and build any dependencies, just like when you run `nix-shell` on your local machine. Dependencies are cached, so the second call to `nix-shell` is very quick compared to the first one. It's pretty cool that we can use this one command to set up an identical build environment to the one on our local machine!
For the WWW site, I wanted a simple theme, without JavaScript or third-party libraries, and preferably with a "retro" feel. Most of this reflects my personal preference towards minimalism, but there are also some restrictions imposed by Sourcehut Pages, namely the CSP header:
Content-Security-Policy: default-src 'self' 'unsafe-eval' 'unsafe-inline'; sandbox allow-forms allow-orientation-lock allow-pointer-lock allow-presentation allow-scripts allow-same-origin;
This prevents you from serving resources from outside the domain. That's fine by me--I like privacy, and the prevalence of CDNs, Web Fonts, and third-party JavaScript undermines that.
Browsing the published themes on Hugo's site, I found several options that were close to what I wanted, but nothing that fit my needs exactly. I broadened my search to GitHub, and somehow I stumbled on risotto, an excellent minimalist "retro" theme with no JavaScript. (Thanks to joeroe for putting it out there.) I could only find one site that used it--this was a brand new theme!
I updated `default.nix` to retrieve my new theme:
hugo-theme-risotto = runCommand "hugo-theme-risotto" {
pinned = builtins.fetchTarball {
# Descriptive name to make the store path easier to identify
name = "hugo-theme-risotto-2020-08-09";
# Gzip of commit hash
url = https://github.com/joeroe/risotto/archive/8d534bcdadbca2bb5343825e119be3e6a710e97a.tar.gz;
# Hash obtained using `nix-prefetch-url --unpack [url]`
sha256 = "1x776i6qnzsyl4vbhm869xx59zspw9m6bx8ms7mhn9vflr9p46x6";
};
# Add path to theme patches below
# Example line: ./patches/hugo-theme-risotto/my-patch.diff
patches = [
./patches/hugo-theme-risotto/logo-orange.diff
];
preferLocalBuild = true;
}
''
cp -r $pinned $out
chmod -R u+w $out
for p in $patches; do
echo "Applying patch $p"
patch -d $out -p1 < "$p"
done
'';
I also had to set the theme options in Hugo's `config.toml`, following the example included with the theme.
As risotto is a new theme, there were still a few minor bugs in the CSS, which I patched locally. Later, when I was confident in the fixes and had more time to test them, I submitted my changes as pull requests on GitHub. I also made some local changes for personalization.
Can this setup solve all of your Markdown-to-Gemini problems? Well, not quite; you must still write "Gemini-aware" Markdown or face awkward formatting issues. For instance, H4 and above are not supported in Gemini, but gmnhg does the best it can by adding a space between the third and fourth `#` character. This is compliant with the Gemini spec (which requires a space there) and it visually indicates a different heading level, but it looks weird in most clients. Because of this, it's best to avoid headings beyond H3.
Another issue is with links. Inline links are not supported in Gemini, so gmnhg leaves the inline link text in place and then adds a link with that same text below the paragraph. If you aren't careful about naming them, you'll end up with a list of nondescript links that have names like "this" and "article." I've actually found this limitation to be a positive, as it forces me to be more thoughtful with my link usage.
Aside from formatting, there are other gaps. The most obvious is that Hugo shortcodes are not currently supported by gmnhg. I don't use shortcodes, but this gap could pose a problem for people who want to publish their existing Hugo sites using gmnhg. I believe that Hugo itself could be used to preprocess shortcodes once it is able to render shortcodes into plain text output. This is not yet possible, but there are some open issues in Hugo that may make it possible in the near future.
As for regrets: there none that I can think of. I learned a great deal from this project, and I'm very happy with the results. I can't even say that I wish I had done this sooner, as the theme and software wouldn't have been available much earlier!
---
Comments? Email the author: mntn at mntn.xyz