💾 Archived View for separateconcerns.com › 2023-04-09-djot.gmi captured on 2024-07-08 at 23:40:27. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2023-04-19)
-=-=-=-=-=-=-
published 2023-04-09
Over 10 years ago [1], I created this blog and wrote the static generator that powers it. I used Markdown for the articles and lunamark [2], a library written by John MacFarlane of CommonMark [3] and Pandoc [4] fame.
Recently, John created djot [5], a markup syntax intended to be an evolution of Markdown. The initial reference implementation [6] was written in Lua and I decided to port my static generator to it.
Converting the Markdown parts of the post themselves was not very hard, the main think I can think of was switching my old posts from intented code blocks - which are not supported by djot - to fenced code blocks. However, my posts were not pure vanilla Markdown. I used two specific features from lunamark and a few extensions of my own.
The first lunamark feature I had to find a replacement for was Lua metadata. djot does not support metadata [7] yet but it does support raw blocks with a format [8] which I could easily abuse.
The second feature is Pandoc-style title blocks, which I could just move to the metadata section [9] so I did that.
I also had a custom block format syntax that I used for three things: raw blocks, syntax highlighting language choice and post descriptions. The first two are natively supported by djot, and I moved the description to the metadata as well.
For those transformations I used Perl one-liners such as this one which converts the metadata format:
for i in separateconcerns/blog/articles/*.md; do
perl -0pe 's/<!\-\-@(.*)\-\->/"```=lua-meta". join("\n", split(\/\n \/, $1)) ."```"/es' \
$i > "$(dirname $i)/$(basename $i .md).dj"
done
Once that was done, it was time to port the generator itself. I started with the HTML part. As I said some of the things I was doing manually in lunamark are supported natively in djot, there were two things left to do: support my metadata blocks and detect posts with code in them so I can include the JavaScript syntax highlighting library only when needed.
Djot has a step when you can get the AST and I could have used it to extract this information, but I decided to use the same method I used with lunamark instead, which is overriding parts of the renderer [10]. To do that I created a small helper library [11] which mostly exposes the StringHandle used internally in djot.lua. I would need it for the next part anyway.
All in all, it was pretty smooth. I just hit a small issue with URL parsing [12] which I worked around for now by using URL encoding in my .dj source files.
Since I did not change my output format much and kept exactly the same metadata, my index and Atom feed generation worked out of the box. But there was still one last thing to do: because this blog is also available on Gemini [13], I had to implement a Gemtext [14] renderer.
In lunamark, there was a generic writer which I overloaded for that purpose. As djot.lua only supports HTML output natively for now, this time I just wrote a renderer [15] from scratch. I could also have gone the Pandoc route as djot includes a custom writer and reader, but I wanted to avoid it and Pandoc does not natively support Gemtext anyway.
All in all, this took a few hours to do, and the blog post you are reading now has been published with this new stack. I like djot, it is an improvement over CommonMark even though it is still has a few rough edges. I like parts of the djot.lua library a bit less, especially the parser [16]. lunamark uses LPeg [17] instead and I find it simpler, but it's still fine.
This experience also conforts me in the choice I made 10 years ago to write my own static generator using a language I know well with few dependencies. It is very low maintenance most of the time compared to the likes of Hugo, Pelican or Jekyll, and it lets me do changes like this easily when I want to.
1: https://blog.separateconcerns.com/2013-01-02-hello-world.html
2: https://github.com/jgm/lunamark
6: https://github.com/jgm/djot.lua
7: https://github.com/jgm/djot/issues/35
8: https://htmlpreview.github.io/?https://github.com/jgm/djot/blob/master/doc/syntax.html#raw-block
9: https://github.com/catwell/cw-web/commit/6518e8284880d6e64ec769ed47517c9dcbba6373
11: https://github.com/catwell/cw-web/blob/master/separateconcerns/blog/cwdjot/init.lua
12: https://github.com/jgm/djot/issues/101#issuecomment-1500966800
13: https://blog.separateconcerns.com/2022-02-05-gemini.html
14: https://gemini.circumlunar.space/docs/gemtext.gmi
15: https://github.com/catwell/cw-web/blob/master/separateconcerns/blog/cwdjot/gemini.lua
16: https://github.com/jgm/djot.lua/blob/5970f1cff2fe6b5905ec1cdeab795d5b5b3b8b97/djot/inline.lua
17: https://github.com/jgm/lunamark/blob/master/lunamark/reader/markdown.lua