Part 2: Automatic Recording of Dependencies on Header Files
Part 3: CFLAGS and friends, config.sh, compile.do
Part 4: CFLAGS and friends, env/VAR, default.run.do
Part 5: Auto-update BUILDDATE in version.h
Part 6: The yacc/bison problem: one call produces two artifacts
Part 7: Test: Generator for N source files
My code featured in this series can be found at
https://git.sr.ht/~ew/ew.redo/
I wanted to have something to simulate a call to a code generator, which will produce a number of artefacts, which in turn are needed to build a (generated) hello world executable. And I wanted to build this thing using redo. How hard can it be?
This was not overly complicated. The generator comes in at 71 lines of code. This comes in a bit smaller than the 73 lines of code I needed in all .do snippets together.
The generator creates N pairs of resource_#.c and resource_#.h files. These are generated at once. The number N is read from a separate file ENV.nmax. These files are generated in a subdirectory, whose name is stored in ENV.tmpdir. The call to the generator is issued by redo itself, just like a call to bison. The state of this affair is recorded in a virtual target file. This file is not used in the code, its content is stamped against redo, such that changes become detectable.
/* resource_0.h */ extern char* message_0;
/* resource_0.c */ char* message_0 = "res 0: Hello, world!";
The generated resources are all the same except for the _#number suffix used in filenames, variable names and strings.
These resource files are then included into hello.c, which in turn is generated, too:
#include <stdio.h>
#include "resource_0.h"
#include "resource_1.h"
int main(int argc, char** argv)
{
printf("%s\n", "start");
printf("%s\n", message_0);
printf("%s\n", message_1);
printf("%s\n", "done.");
}
It should be easy to see, that all the generated resource files are going to be needed. Please note, that in order to build hello.o from hello.c all resource_#.h files must be generated at this point. This requires a separate hello.o.do file including some magic to record these dependencies:
# hello.o.do
Nmax=$(cat ENV.nmax)
# the header files must be generated before gcc is called!
HEADERS="" ; N=0 ; while [ $N -lt $Nmax ] ; do
HEADERS="$HEADERS resource_${N}.h"
let N=N+1
done
redo-ifchange hello.c $HEADERS
gcc -MMD -MF hello.d -I . -o $3 -c hello.c
read DEPS < hello.d
redo-ifchange ${DEPS#*:}
The generator is called in a .do snippet called run.generator.do.
# run.generator.do
if [ ! -e ENV.tmpdir ]
then
echo "file ENV.tmpdir not found"
exit 99
fi
tmpdir=$(cat ENV.tmpdir)
if [ ! -d ${tmpdir} ]
then
mkdir -p ./${tmpdir}
fi
redo-ifchange generator_code.sh
./generator_code.sh
sync
md5sum $tmpdir/hello.c $tmpdir/resource_*.h $tmpdir/resource_*.c > "$3"
#redo-always
redo-stamp < "$3"
The script takes care of the wanted temporary directory, creating it as needed. It then calls the generator and records the state of affairs in its target run.generator. Using wildcards in the md5sum call might record leftover files, if Nmax is reduced. redo clean is your friend.
In order to avoid unneeded rebuilds, the source files consumed by the compiler are copied from the temporary directory. This allows redo to store individual state about every file and flag things as unchanged. There are two separate snippets to copy .c and .h files accordingly:
# default.c.do tmpdir=$(cat ENV.tmpdir) redo-ifchange run.generator cp -a "$tmpdir"/$2.c "$3" redo-always redo-stamp < $3
# default.h.do tmpdir=$(cat ENV.tmpdir) redo-ifchange run.generator cp -a "$tmpdir"/$2.h "$3" redo-always redo-stamp < $3
# all.do redo-ifchange hello
# clean.do
tmpdir=$(cat ENV.tmpdir)
rm -f hello *.o *.d hello.c resource_*.[cho] run.generator
rm -fr "${tmpdir}"
# default.o.do
redo-ifchange $2.c
gcc -MMD -MF $2.d -I . -o $3 -c $2.c
read DEPS <$2.d
redo-ifchange ${DEPS#*:}
The difference between hello.o.do and default.o.do is, that in hello.o.do a dependency to all resource_#.h files is explicitly listed. Otherwise these files do not exist before gcc is called.
# hello.c.do tmpdir=$(cat ENV.tmpdir) redo-ifchange run.generator cp -a "$tmpdir"/hello.c "$3" redo-always redo-stamp < $3
The difference between hello.c.do and default.c.do is that there is no dependency on hello.h ($2.h).
# hello.do
Nmax=$(cat ENV.nmax)
OBJS="hello.o"
N=0 ; while [ $N -lt $Nmax ] ; do
OBJS="$OBJS resource_${N}.o"
let N=N+1
done
redo-ifchange run.generator $OBJS
gcc $OBJS -o $3
This experiment confirmed, that I have gathered enough puzzle pieces to actually understand this sort of build.
Finally I would like to recommend the use of redo-dot. The resulting diagram has allowed me to clean out a number of leftovers from experimentation. If the diagram looks crook, even if you remove the dottet lines, then the build probably is crook, too.
shell$ redo-dot | grep -v dotted > diagram.dot dot -Tpng diagram.dot > diagram.png
/file/20230128-redo7-diagram.png
Cheers,
~ew
