Part 2: Automatic Recording of Dependencies on Header Files
Part 3: CFLAGS and friends, config.sh, compile.do
Part 4: CFLAGS and friends, env/VAR, default.run.do
Part 5: Auto-update BUILDDATE in version.h
Part 6: The yacc/bison problem: one call produces two artifacts
Part 7: Test: Generator for N source files
My code featured in this series can be found at
https://git.sr.ht/~ew/ew.redo/
How about updating something like BUILDDATE or GITVERSION automatically in some generated version.h file? Wouldn't that be nice? Or in more technical terms:
How hard can it be?
Turns out, it was harder than I expected, because I stomped on a bug lurking in goredo. Good news: After reporting the error, Sergey Matveev has fixed it in version 1.29.0.
The example below was basically recycled fron the documentation of Avery Pennaruns redo:
https://redo.readthedocs.io/en/latest/cookbook/defaults/
https://github.com/apenwarr/redo/
# version.do git rev-parse --short HEAD > "$3" redo-always redo-stamp <"$3"
The snippet is quite simple, the important lines are the last two.
This way any generated content can be checked for changes.
# date.do date +'%Y%m%d_%H%M%S' >"$3" redo-always redo-stamp <"$3"
Same sort of script. It should be clear that including the seconds in the timestamp will make this target change pretty much always.
So we want to see that the content of file version does not change on redo, whereas the content of file date does change:
shell$ cat date 20221217_141209 shell$ sed -n '/Type: stamp/,/Hash:/p' .redo/date.rec Type: stamp Hash: 8d46aa5ac04e9d391280bb0f3618a32112bf10a70ffcb08b38df29aaa41f1ce4 shell$ redo date redo date (0.006s) shell$ cat date 20221217_141854 shell$ sed -n '/Type: stamp/,/Hash:/p' .redo/date.rec Type: stamp Hash: bcd9ded266a2a5fde9dd5c2742a6e13fd23e66f30549f2a39b26d10f37cefb9c shell$ cat version dff1aa5 shell$ sed -n '/Type: stamp/,/Hash:/p' .redo/version.rec Type: stamp Hash: 07675f7c17ba6e19a41249e0e2e5daf81c7950a4f38ceb007a16ca33afd5d6e7 shell$ redo version redo version (0.006s) shell$ cat version dff1aa5 shell$ sed -n '/Type: stamp/,/Hash:/p' .redo/version.rec Type: stamp Hash: 07675f7c17ba6e19a41249e0e2e5daf81c7950a4f38ceb007a16ca33afd5d6e7
Works as advertised.
From these two files, we create a new file include/version.h. So we have a template include/version.h.in and a script include/version.h.do to expand the template and produce the desired result.
/* version.h.in */ #ifndef __VERSION_H #define __VERSION_H #define PRJ_VERSION "%%VERSION%%" #define PRJ_BLDDATE "%%DATE%%" #endif // __VERSION_H
The corresponding redo snippet "include/version.h.do" edits the template and produces the desired file "include/version.h"
# include/version.h.do
redo-ifchange ../date ../version version.h.in
VERSION=$(cat ../version)
DATE=$(cat ../date)
cat "$2".in |
sed -e "s/%%VERSION%%/${VERSION}/g" \
-e "s/%%DATE%%/${DATE}/g" \
> "$3"
redo-always
redo-stamp <"$3"
Again file include/version.h is always generated, but is considered unchanged, if its content is unchanged. This can be tested by removing the line, which edits %%DATE%%, or by removing the time part of the output in date.do. In that case nothing is rebuild on subsequent calls to redo.
So we instrument hello.c to use version.h accordingly:
#include <stdio.h>
#include "resources.h"
#include "version.h"
int main(int argc, char** argv)
{
printf("%s %s (compiled %s)\n", "hello", PRJ_VERSION, PRJ_BLDDATE );
printf("%s\n", message);
}
With both variables being considered, include/version.h changes and hello.o and thus hello is being rebuild:
shell$ redo clean; rm -fr .redo/ include/.redo redo clean (0.003s) shell$ redo include/version.h redo . ../date (0.006s) redo . ../version (0.007s) redo include/version.h (0.043s) shell$ redo redo . . link.run (default.run.do) (0.017s) redo . . . compile.run (default.run.do) (0.015s) redo . . resources.o (default.o.do) (0.083s) redo . date (0.007s) redo . version (0.007s) redo . include/version.h (0.009s) redo . . hello.o (default.o.do) (0.140s) redo . hello (0.185s) redo all (0.197s)
Unfortunately there is a chicken-and-egg problem in here. Building include/version.h explicitly will make the remaining build succeed. However, starting from scratch fails:
shell$ redo clean; rm -fr .redo/ include/.redo
redo clean (0.003s)
shell$ redo
redo . . link.run (default.run.do) (0.019s)
redo . . . compile.run (default.run.do) (0.015s)
hello.c:3:10: fatal error: include/version.h: No such file or directory
3 | #include "include/version.h"
| ^~~~~~~~~~~~~~~~~~~
compilation terminated.
./compile.run: line 4: hello.d: No such file or directory
redo . . resources.o (default.o.do) (0.055s)
redo . . hello.o (default.o.do) (0.081s)
/usr/bin/ld: cannot find hello.o: No such file or directory
collect2: error: ld returned 1 exit status
err . hello (0.110s): exit status 1
err all (0.114s): exit status 1
This problem is not so hard to understand: The generated file include/version.h is referenced in hello.c. So include/version.h is a prerequisite for hello.o. Moreover, it is automatically detected and added via compile.run. However, compile.run or more precisely "gcc -MMD" will fail if include/version.h is not yet available. So there is a chicken-or-egg problem. That means we have to add this dependency explicitly. It should not come as a surprise, that adding this to default.o.do is questionable, because now all .c files whereever they reside in this build tree will depend on include/version.h. The solution is to add an extra snippet for hello.o:
# hello.o.do redo-ifchange compile.run include/version.h $2.c ./compile.run $2.c hello $3
With that in place, and goredo of version 1.29.0 or later, things will start to work:
shell$ redo -version goredo 1.30.0 built with go1.19.4 shell$ redo clean; rm -fr .redo/ include/.redo redo clean (0.003s) shell$ redo redo . . link.run (default.run.do) (0.018s) redo . . . compile.run (default.run.do) (0.018s) redo . . . . ../date (0.007s) redo . . . . ../version (0.007s) redo . . . include/version.h (0.064s) redo . . resources.o (default.o.do) (0.086s) redo . . hello.o (0.140s) redo . hello (0.185s) redo all (0.197s) shell$ cat date 20230122_192425 shell$ ./hello hello e747669 (compiled 20230122_192425) Hello, world! shell$ redo redo . date (0.006s) redo . version (0.007s) redo . include/version.h (0.009s) redo . . hello.o (0.031s) redo . . . hello (0.028s) redo . . . . all (0.004s) shell$ cat date 20230122_192436 shell$ ./hello hello e747669 (compiled 20230122_192436) Hello, world!
The bug mentioned above had to do with a target (version.h) being triggered twice in one redo run. The second invocation would always fail with
err . version.h (0.021s): $1 was explicitly touched
Thanks to Sergey for fixing this immediately.
Cheers,
~ew