💾 Archived View for ew.srht.site › en › 2023 › 20230128-redo-7.gmi captured on 2024-02-05 at 09:33:51. Gemini links have been rewritten to link to archived content

-=-=-=-=-=-=-

2023-01-28

Tools: redo (part 7) The N artefact problem, a minimal example

tags: software

Content

Part 0: Intro

Part 1: Hello, world!

Part 2: Automatic Recording of Dependencies on Header Files

Part 3: CFLAGS and friends, config.sh, compile.do

Part 4: CFLAGS and friends, env/VAR, default.run.do

Part 5: Auto-update BUILDDATE in version.h

Part 6: The yacc/bison problem: one call produces two artifacts

Part 7: Test: Generator for N source files

My code featured in this series can be found at

https://git.sr.ht/~ew/ew.redo/

I wanted to have something to simulate a call to a code generator, which will produce a number of artefacts, which in turn are needed to build a (generated) hello world executable. And I wanted to build this thing using redo. How hard can it be?

This was not overly complicated. The generator comes in at 71 lines of code. This comes in a bit smaller than the 73 lines of code I needed in all .do snippets together.

Generator

The generator creates N pairs of resource_#.c and resource_#.h files. These are generated at once. The number N is read from a separate file ENV.nmax. These files are generated in a subdirectory, whose name is stored in ENV.tmpdir. The call to the generator is issued by redo itself, just like a call to bison. The state of this affair is recorded in a virtual target file. This file is not used in the code, its content is stamped against redo, such that changes become detectable.

resource_#.c resource_#.h

/* resource_0.h */
extern char* message_0;

/* resource_0.c */
char* message_0 = "res 0: Hello, world!";

The generated resources are all the same except for the _#number suffix used in filenames, variable names and strings.

hello.c

These resource files are then included into hello.c, which in turn is generated, too:

#include <stdio.h>
#include "resource_0.h"
#include "resource_1.h"
int main(int argc, char** argv)
{
    printf("%s\n", "start");
    printf("%s\n", message_0);
    printf("%s\n", message_1);
    printf("%s\n", "done.");
}

It should be easy to see, that all the generated resource files are going to be needed. Please note, that in order to build hello.o from hello.c all resource_#.h files must be generated at this point. This requires a separate hello.o.do file including some magic to record these dependencies:

# hello.o.do
Nmax=$(cat ENV.nmax)
# the header files must be generated before gcc is called!
HEADERS="" ; N=0 ; while [ $N -lt $Nmax ] ; do
   HEADERS="$HEADERS resource_${N}.h"
   let N=N+1
done
redo-ifchange hello.c $HEADERS
gcc  -MMD -MF hello.d  -I . -o $3 -c hello.c
read DEPS < hello.d
redo-ifchange ${DEPS#*:}

Calling the generator from redo

The generator is called in a .do snippet called run.generator.do.

# run.generator.do
if [ ! -e ENV.tmpdir ]
then
    echo "file ENV.tmpdir not found"
    exit 99
fi
tmpdir=$(cat ENV.tmpdir)
if [ ! -d ${tmpdir} ]
then
    mkdir -p ./${tmpdir}
fi

redo-ifchange generator_code.sh
./generator_code.sh
sync
md5sum $tmpdir/hello.c $tmpdir/resource_*.h $tmpdir/resource_*.c > "$3"
#redo-always
redo-stamp < "$3"

The script takes care of the wanted temporary directory, creating it as needed. It then calls the generator and records the state of affairs in its target run.generator. Using wildcards in the md5sum call might record leftover files, if Nmax is reduced. redo clean is your friend.

Copy on change

In order to avoid unneeded rebuilds, the source files consumed by the compiler are copied from the temporary directory. This allows redo to store individual state about every file and flag things as unchanged. There are two separate snippets to copy .c and .h files accordingly:

# default.c.do
tmpdir=$(cat ENV.tmpdir)

redo-ifchange run.generator
cp -a "$tmpdir"/$2.c "$3"
redo-always
redo-stamp < $3

# default.h.do
tmpdir=$(cat ENV.tmpdir)

redo-ifchange run.generator
cp -a "$tmpdir"/$2.h "$3"
redo-always
redo-stamp < $3

El Resto

# all.do
redo-ifchange hello

# clean.do
tmpdir=$(cat ENV.tmpdir)
rm -f hello *.o *.d hello.c resource_*.[cho] run.generator
rm -fr "${tmpdir}"

# default.o.do
redo-ifchange $2.c
gcc  -MMD -MF $2.d  -I . -o $3 -c $2.c
read DEPS <$2.d
redo-ifchange ${DEPS#*:}

The difference between hello.o.do and default.o.do is, that in hello.o.do a dependency to all resource_#.h files is explicitly listed. Otherwise these files do not exist before gcc is called.

# hello.c.do
tmpdir=$(cat ENV.tmpdir)

redo-ifchange run.generator
cp -a "$tmpdir"/hello.c "$3"
redo-always
redo-stamp < $3

The difference between hello.c.do and default.c.do is that there is no dependency on hello.h ($2.h).

# hello.do
Nmax=$(cat ENV.nmax)
OBJS="hello.o"
N=0 ; while [ $N -lt $Nmax ] ; do
    OBJS="$OBJS resource_${N}.o"
    let N=N+1
done
redo-ifchange run.generator $OBJS
gcc $OBJS -o $3

Conclusion

This experiment confirmed, that I have gathered enough puzzle pieces to actually understand this sort of build.

Finally I would like to recommend the use of redo-dot. The resulting diagram has allowed me to clean out a number of leftovers from experimentation. If the diagram looks crook, even if you remove the dottet lines, then the build probably is crook, too.

shell$ redo-dot | grep -v dotted  > diagram.dot
dot -Tpng diagram.dot > diagram.png

/file/20230128-redo7-diagram.png

Cheers,

~ew

Home