💾 Archived View for gemi.dev › gemlog › 2023-01-19-build-script.gmi captured on 2023-11-04 at 11:40:29. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2023-01-29)
-=-=-=-=-=-=-
2023-01-18 | #tag | @Acidus
Build scripts are super helpful. By "build scripts" I mean Makefiles, a publish.sh shell script, really anything that automates building or creating something. Usually that's a program, but I find them helpful even for things that are not strictly speaking development work.
For example, I have been porting the content from some Apple history projects to Markdown. I also am publishing them as gemtext on my capsule. This is an iterative process, so generating the gemtext is something I do as I add more content.
Apple History and the Creation of the Macintosh
I know what you are thinking. Converting Markdown to gemtext? that's so easy! Just a run of md2gemini right? And that's why most people don't write a build script. They think its *just* one thing, or one command. But when you break it down, its actually more.
Lets see a non-code example. Those 2 content projects consist of Markdown documents and images, and the structure of the projects looks like this:
Let's see how I have evolved the build scripts over time to convert the Markdown and images into gemtext for my capsule.
First, I want to make sure that when I'm generating I'm not going to accidentally clobber anything. I should put the converted text into an output directory. I also don't want old or out-of-date copies to be there, so this should start clean and fresh each time. So my build script `publish.sh` looks like this:
# remove any existing output directory, and create a new clean one rm -rf output mkdir output # copy all the Markdown, and the images/ directory into output/ cp -R Posts/* output/.
Now, all my actions will operate on the contents in output/, and I don't have to worry about messing up any of the source content.
This is pretty straight forward. I'm running a forked version of md2gemini that allows me to convert image links in the Markdown to link lines in gemtext, but doesn't convert the hyperlinks in the Markdown.
The command is:
md2gemini --write --dir output/ --frontmatter --img-tag='' --links images-only output/*.md
This command will generate .gmi files in output/ for all the Markdown files (*.md) in output/ as well. Of course, I don't need those md files anymore, so I can delete them:
md2gemini --write --dir output/ --frontmatter --img-tag='' --links images-only output/*.md rm output/*.md
Unfortunately, md2gemini generates files that use CRLF to end each line, even if you run it on a system like Linux or macOS which just uses LF to end a line. If I edit these files, and end up with some lines ending with CRLF and some with LF, this can cause problems with some gemini clients or text editors. So I convert all the gemtext files in output/ to only LF for newlines:
md2gemini --write --dir output/ --frontmatter --img-tag='' --links images-only output/*.md rm output/*.md dos2unix -q output/*.gmi
Now I just have clean gemtext files (*.gmi) in output/ as well as images those files reference in output/images/.
When porting the source content to Markdown, I try and keep the highest quality and resolution images possible. This is because I'm trying to preserve this content before it rots away on the web. However a 1.5 MG JPEG of the cover of Time Magazine from the early 1980s isn't really appropriate for Geminispace. I should resize it. Perhaps I want to only have it be no more than 800 pixels wide or 800 pixels tall. I can use ImageMagick's `mogrify` command to bulk resize the images.
mogrify -strip -quality 80 -resize 800x800\> output/images/*
This also strips the meta data from the images, to make them smaller as well. Because I'm operating in the output directory, I can do anything I want without fear of messing up the source images.
So far, my publish.sh looks like this:
# remove any existing output directory, and create a new clean one rm -rf output mkdir output # copy all the Markdown, and the images/ directory into output/ cp -R Posts/* output/. # Convert Markdown to gemtext md2gemini --write --dir output/ --frontmatter --img-tag='' --links images-only output/*.md rm output/*.md dos2unix -q output/*.gmi # optimize images mogrify -strip -quality 80 -resize 800x800\> output/images/*
But because it's a script I can always add more things:
You can see the actual build script I use below (I happen to convert it to make, but it was a shell script)
Makefile for Folklore.org Mirror publishing
This is how automation with a built script frees you, even for non-code projects. Before I was trying to save images and make sure they were the appropriate format for gemini. Now, I don't care. I don't need to worry about it. I have a system that will always copy and optimize the images, no matter what.
It was only after I had a built script that I realized how often I needed to do the steps. Since it was manual and ad-hoc before, I would do it more rarely, instead waiting for a large number of changes to "build up." With a built script I go so much faster. Fix a typo? Rebuild the content. Add a higher quality image? Rebuild the content. Once its automated, the cost to use it is effectively zero.
Also important is that consistency. Once its automated, the same thing happens every time, without you needing to remember the steps. And speaking of remembering, build scripts make it easy to return to a project later the future. I don't have to remember all the steps needed to publish my Apple II history project. I just run publish.sh.
Everyone thinks the project they are working on is "too small" for a build script. But chances are, if you are working on a project, you have probably already written a build script without knowing it. You probably have some terminal open and you keep pressing "up" a few times to go back in your command history and re-run something. Maybe you run 1 command, and then run a second command over and over again as you are iterating on something trying to get it right. Just copy that command into a publish.sh file! In fact, if I ever find myself repeatedly needing to execute 1 or 2 commands over and over I just go ahead and make a shell script for it. These quickly evolve into publishing scripts.