💾 Archived View for heavysquare.com › notes › FFE5-builds-systems-detecting-noop.txt captured on 2021-12-04 at 18:04:22.

View Raw

More Information

⬅️ Previous capture (2021-12-03)

-=-=-=-=-=-=-


~~ 2021-03-17T10:14:15+01:00

One way to think about build systems is to pretend they're DAGs,
directed acyclic graphs. Nodes are build artifacts, edges are actions
that supposedly update.. Well make this a hypergraph in the sense that
an "edge" is having source nodes (plural) and target nodes. So an edge
is an action which take sources and produce the target artifacts. The
build is ready.. when?

Make works with timestamps, so if for all edges, the earliest target
timestamp is after the latest source timestamp, we're ready.

What happens if I have source code and I add a comment? Let's say it's
C. No need to recompile, to rerun all the tests, etc. At this point I'd
like to mention that this improvement works only with immutable action,
so that the output is only a function of the input.. no timestamp added
to the output etc. So instead of observing timestamps of the data, you
can observe the hash of if, and maintain a list of data hashes for all
the edges, like all input data hash, output data hash when computed.

You're ready with the build if for all edges there's a stored pair of
hashes and when you hash the input, it's the stored input hash. In case
your build is having the C preprocessed files as a step, when you remove
a comment, this improved build system will recompile the source file into
the preprocessed, but then stop because that's the same as before. Nice.


lurking in the background.

Sandboxing is when your build system, when processing an edge, an action
that produces outputs from inputs, moves the inputs to a separate location
and run the edge there. This way, the sources should be at least the ones
needed and you won't forget to list your inputs. That's bad because when
you have an input that should normally trigger some actions, it won't,
as it's not listed in the graph.

How to fuck all these up. Like maven/java. -> with SNAPSHOT versions. This
is a form of mutable data, so maven will recheck SNAPSHOT versioned
artifacts every once in a while if there's a new one. But, you can turn
that off with the --offline flag.

Another feature that's really handy for CI. You don't want to rerun all
your tests if you only changed the source of one single test. This is
a problem when you have tests suites running for hours.