💾 Archived View for log.pfad.fr › 2023 › devf-axe-sharpening captured on 2023-11-04 at 11:06:41. Gemini links have been rewritten to link to archived content

-=-=-=-=-=-=-

Sharpening my developer axe (introducing devf for live reloading)

I write quite a lot of software in Go, mostly web-based. This means that I have my code editor and my web browser opened in parallel, to visually check what my code produces. Naively it involves the following manual steps:

save the code
compile the binary/generate the static site
reload the page in the browser

Since other developers already had this issue a couple of open-source solutions are already available. Until now I have been using the combo modd+devd, which automates the 2 last steps above (compile and reload):

modd: flexible developer tool that runs processes and responds to filesystem changes

devd: local webserver for developers

Configuration example for modd:

# Exclude all test files of the form *_test.go

	*/*.go !**/*_test.go {

    daemon: go run ./cmd/server
}

# devd: reload on sighup
server.is_running internal/**/*.gohtml internal/assets/*.* {
    daemon: go run github.com/cortesi/devd/cmd/devd -qL http://localhost:8989
}

The first block launches `go run ./cmd/server` when any (non-test) go file changes.

In the `./cmd/server/main.go`, I start a local web server, which writes an empty `server.is_running` file when the server is ready to accept connections. When modd detects that this file has been updated, it sends a sighup to devd, which reloads the browser (this also happens whenever I update a gohtml template file).

This setup served me well for a time, but I encountered some frustration lately:

devd: sometimes the browser loading would hang (just after the reload trigger)
devd: when upstream does not accept the connection (panic for instance), the error page display would not live-reload on subsequent saves.
modd has a backoff "feature" which imposes up to 8 seconds delay when the go files repeatedly change within a short time frame

Looking at the codebase of devd, I saw no recent push activity (last commit more than 3 years ago) and quite a lot of code (52 files, 12 packages). I then had the idea to develop my own devd.

Introducing devf

code.pfad.fr/devf (live-reload web server for developers)

devf attempts to do one thing and do it well: serve as a reverse proxy while injecting some JS to support livereloading.

The reloading must be externally triggered, either by writing to its standard input or sending predefined signals. It can either server local files or proxy to some upstream server.

Since I intentionally skipped many features of devd (throttling, tls, watcher, basic auth, CORS...), it needs to be wired up with other tools (like modd or inotifywait). The `livereload` subpackage exposes an http.Handler that can be imported if one wants to build on top of it.

Example usage for static website

For this blog, I have the following Makefile:

build:
	# compile the blog, put it to dist and write the .dist.is_done file
	@rm -rf dist
	./kiln build
	@touch .dist.is_done

dev: build
	$(MAKE) --no-print-directory -j2 watch serve-html

watch:
	@inotifywait -m -e close_write -r --exclude "(dist|\.git)" . | xargs -I{} -d "\n" $(MAKE) --no-print-directory build

serve-html:
	inotifywait -m -e close_write .dist.is_done | go run code.pfad.fr/devf@latest -addr=:8087 dist/html

When I run `make dev` two tasks are launched in parallel, after an initial build:

`watch` will launch the `make build` task if any file changes (outside of dist and .git)
`serve-html` will serve dist/html and trigger a live-reload anytime the file `.dist.is_done` changes

Both tasks work by piping the output of `inotifywait` (one line on every file-change event) to other tools. I consider this to be a nice example of composibility in linux.

Example usage for local server

I found this Makefile much harder to write since it needs to interrupt a long-running process whenever a file changes.

dev:
	$(MAKE) --no-print-directory -j2 dev-devf dev-backend

dev-devf:
	inotifywait -m -e close_write --include "\.(is_running|gohtml|js|css)$" -r . | \
	go run code.pfad.fr/devf -addr localhost:8000 http://localhost:8989

dev-backend:
	pid="nonexisting" ; \
	(echo "initial generation" && inotifywait -m -e close_write --include "\.(go)$" -r .) | \
	while read -r path; do \
		echo "$path and $(timeout 0.1 cat | wc -l) other changes" ; \
		pkill -TERM -P $pid ; \
		wait $pid ; \
		( \
			go run ./cmd/server \
		) & \
		pid=$! ; \
	done

When I run `make dev` two tasks are launched in parallel:

`dev-devf` will proxy http://localhost:8000 to http://localhost:8989, inject the livereload script and trigger the reload whenever a is_running, gohtml, js or css file changes.
`dev-backend` will launch `go run ./cmd/server` in the background, and save its pid. Whenever a go file changes, the process with the saved pid will be killed and `go run` relaunched.

The `dev-backend` has a few quirks:

it sets an initial (invalid) pid (`pkill` and `wait` complain a bit about this, but it works)
it echos a dummy line at the beginning to ensure `go run ./cmd/server` starts immediately (otherwise it would only start on the first file change)
it debounces the inotifywait events for 0.1 second (when code gets generated, many files are changed at the same time, making inotifywait write as many lines)

I would be grateful for any tips, if anyone knows how to simplify the `dev-backend` task, to remove some of those quirks!

Conclusion

Rewriting a tool that I use regularly was very interesting and a lot of fun. It allowed me to learn quite a lot about using pipes and inotify on Linux for my various workflows (even if they don't need live reloading).

Thanks to using the commands above, I don't have to leave my editor when coding: the code gets compiled and my browser reloads immediately.

Feel free to try it out and let me know how it works for you!

code.pfad.fr/devf (live-reload web server for developers)

📅 2023-06-26

Back to the index

Send me a comment or feedback