Plan 9 Mkfiles
Bob Flandrena
Introduction
Every Plan 9 source directory contains a file, called mkfile, specifying the rules for building the executable or library that is the product of the directory. Mk(1) interprets the rules in the file, calculates the dependencies, and executes an rc(1) script to construct the product. If necessary components are supplied by neighboring directories or sub-directories, the mkfiles in those directories are first executed to build the components before the local construction proceeds.
Most application source directories produce one of four types of product: a single executable, several executables, a local library, or a system library. Four generic mkfiles define the normal rules for building each type of product. The simplest mkfiles need only list the components and include the appropriate generic mkfile to do the work. More complex mkfiles may supply additional rules to augment, modify, or override the generic rules.
Using a Mkfile
To build a product, change to the directory containing its source and invoke mk with the appropriate target as an argument. All mkfiles provide the following standard targets:
If no target is specified on the mk command line, the all target is built by default. In a directory producing multiple executables, there is no default target.
In addition to the five standard targets, additional targets may be supplied by each generic mkfile or by the directory’s mkfile.
The environment variable NPROC is set by the system to the number of available processors. Setting this variable, either in the environment or in a mkfile, controls the amount of parallelism in the build. For example, the command
NPROC=1 mk
restricts a build to a single thread of execution.
Creating a Mkfile
The easiest way to build a new mkfile is to copy and modify an existing mkfile of the same type. Failing that, it is usually possible to create a new mkfile with minimal effort, since the appropriate generic mkfile predefines the rules that do all the work. In the simplest and most common cases, the new mkfile need only define a couple of variables and include the appropriate architecture-specific and generic mkfiles.
There are four generic mkfiles containing commonly used rules for building a product: mkone, mkmany, mklib, and mksyslib. These rules perform such actions as compiling C source files, loading object files, archiving libraries, and installing executables in the bin directory of the appropriate architecture. The generic mkfiles are stored in directory /sys/src/cmd. Mkfile mkone builds a single executable, mkmany builds several executables from the source in a single directory, and mklib and mksyslib, maintain local and system libraries, respectively. The rules in the generic mkfiles are driven by the values of variables, some of which must be set by the product mkfile and some of which are supplied by the generic mkfile. Variables in the latter class include:
The following variables are set by the product mkfile and used by the generic mkfile. Any may be empty depending on the specific product being made.
Mkfile Organization
All mkfiles share the following common structure:
</$objtype/mkfile # architecture-dependent definitions
variable definitions # TARG, OFILES, HFILES, etc.
</sys/src/cmd/generic # mkone, mkmany, mklib, or mksyslib
variable overrides # CFLAGS, objtype, etc.
extra rules # overrides, augmented rules, additional targets
Note that the architecture-dependent mkfiles include file /sys/src/mkfile.proto for system-wide variables that are common to all architectures.
The variables driving the expansion of the generic mkfile may be specified in any order as long as they are defined before the inclusion of the generic mkfile. The value of a variable may be changed by assigning a new value following the inclusion of the generic mkfile, but the effects are sometimes counter-intuitive. Such variable assignments do not apply to the target and prerequisite portions of any previously defined rules; the new values only apply to the recipes of rules preceding the assignment statement and to all parts of any rules following it.
The rules supplied by the generic mkfile may be overridden or augmented. The new rules must be specified after the inclusion of the generic mkfile. If the target and prerequisite portion of the rule exactly match the target and prerequisite portion of a previously defined rule and the new rule contains a recipe, the new rule replaces the old one. If the target of a new rule exactly matches the target of a previous rule and one or more new prerequisites are specified and the new rule contains no recipe, the new prerequisites are added to the prerequisites of the old rule.
Following sections discuss each generic mkfile in detail.
Mkone
The mkone generic mkfile contains rules for building a single executable from one or more files in a directory. The variable TARG specifies the name of the executable and variables OFILES and YFILES specify the object files and yacc source files used to build it. HFILES contains the names of the local header files included in all source files. BIN is the name of the directory where the executable is installed. LIB contains the names of local libraries used by the linker. This variable is rarely needed as libraries referenced by a #pragma directive in an associated header file, including all system libraries, are automatically searched by the loader.
If mk is executed without a target, the all target is built; it produces an executable in $O.out. Variable HFILES identifies the header files that are included in all or most or the C source files. Occasionally, a program has other header files that are only used in some source files. A header can be added to the prerequisites for those object files by adding a rule of the following form following the inclusion of generic mkfile mkone:
file.$O: header.h
The mkfile for a directory producing a single executable using the normal set of rules is trivial: a list of some files followed by the inclusion of mkone. For example, /sys/src/cmd/diff/mkfile contains:
< /$objtype/mkfile
TARG=diff
OFILES=\
diffdir.$O\
diffio.$O\
diffreg.$O\
main.$O\
HFILES=diff.h
BIN=/$objtype/bin
</sys/src/cmd/mkone
The more complex mkfile in /sys/src/cmd/awk overrides compiler and loader variables to select the ANSI/POSIX Computing Environment with appropriately defined command line variables. It also overrides the default yacc rule to place the output soure in file awkgram.c and the clean and nuke rules, so it can remove the non-standard intermediate files. Finally, the last three rules build a version of maketab appropriate for the architecture where the mk is being run and then executes it to create source file proctab.c:
</$objtype/mkfile
TARG=awk
OFILES=re.$O\
lex.$O\
main.$O\
parse.$O\
proctab.$O\
tran.$O\
lib.$O\
run.$O\
awkgram.$O\
HFILES=awk.h\
y.tab.h\
proto.h\
YFILES=awkgram.y
BIN=/$objtype/bin
</sys/src/cmd/mkone
CFLAGS=-c -D_REGEXP_EXTENSION -D_RESEARCH_SOURCE \
-D_BSD_EXTENSION -DUTF
YFLAGS=-S -d -v
CC=pcc
LD=pcc
cpuobjtype=‘{sed -n ’s/^O=//p’ /$cputype/mkfile}
y.tab.h awkgram.c: $YFILES
$YACC -o awkgram.c $YFLAGS $prereq
clean:V:
rm -f *.[$OS] [$OS].out [$OS].maketab y.tab.? y.debug\
y.output $TARG
nuke:V:
rm -f *.[$OS] [$OS].out [$OS].maketab y.tab.? y.debug\
y.output awkgram.c $TARG
proctab.c: $cpuobjtype.maketab
./$cpuobjtype.maketab >proctab.c
$cpuobjtype.maketab: y.tab.h maketab.c
objtype=$cputype
mk maketab.$cputype
maketab.$cputype:V: y.tab.h maketab.$O
$LD -o $O.maketab maketab.$O
Mkmany
The mkmany generic mkfile builds several executables from the files in a directory. It differs from the operation of mkone in three respects: TARG specifies the names of all executables, there is no default command-line target, and additional rules allow a single executable to be built or installed.
The TARG variable specifies the names of all executables produced by the mkfile. The rules assume the name of each executable is also the name of the file containing its main function. OFILES specifies files containing common subroutines loaded with all executables. Consider the mkfile:
</$objtype/mkfile
TARG=alpha beta
OFILES=common.$O
BIN=/$objtype/bin
</sys/src/cmd/mkmany
It assumes the main functions for executables alpha and beta are in files alpha.$O and beta.$O and that both programs use the subroutines in file common.$O. The all target builds all executables, leaving each in a file with a name of the form $O.progname where progname is the name of the executable. In this example the all target produces executables $O.alpha and $O.beta.
The mkmany rules provide additional targets for building a single executable:
Mklib
The mklib generic mkfile builds a local library. Since this form of mkfile constructs no executable, the TARG and BIN variables are not needed. Instead, the LIB variable specifies the library to be built or updated. Variable OFILES contains the names of the object files to be archived in the library. The use of variables YFILES and HFILES does not change. When possible, only the out-of-date members of the library are updated.
The variable LIBDIR contains the name of the directory where the library is installed; by default it selects the current directory. It can be overridden by assigning the new directory name after the point where mklib is included.
The clean target removes object files and yacc intermediate files but does not touch the library. The nuke target removes the library as well as the files removed by the clean target. The command
mk -s clean all
causes the existing library to be updated, or created if it doesn’t already exist. The command
mk -s nuke all
forces the library to be rebuilt from scratch.
The mkfile from /sys/src/cmd/upas/libString contains the following specifications to build the local library libString.a$O for the object architecture referenced by $O:
</$objtype/mkfile
LIB=libString.a$O
OFILES= s_alloc.$O\
s_append.$O\
s_array.$O\
s_copy.$O\
s_getline.$O\
s_grow.$O\
s_nappend.$O\
s_parse.$O\
s_read.$O\
s_read_line.$O\
s_tolower.$O\
</sys/src/cmd/mklib
nuke:V:
mk clean
rm -f libString.a[$OS]
The override of the rule for target nuke removes the libraries for all architectures as opposed to the default recipe for this target which removes the library for the current architecture.
Mksyslib
The mksyslib generic mkfile is similar to the mklib mkfile except that it operates on a system library instead of a local library. The install and all targets are the same; since there is no local copy of the library, all updates are performed on the installed library. The rule for the nuke target is identical to that of the clean target; unlike the nuke target for local libraries, the library is never removed.
No attempt is made to determine if individual library members are up-to-date; all members of a library are always updated. Special targets support manipulation of a single object file; the target objfile updates file objfile.$O in the library of the current architecture and the target objfile.all updates objfile.$O in the libraries of all architectures.
Overrides
The rules provided by a generic mkfile or the variables used to control the evaluation of those rules may be overridden in most circumstances. Overrides must be specified in the product mkfile after the point where the generic mkfile is included; in general, variable and rule overrides occupy the end of a product mkfile.
The value of a variable is overridden by assigning a new value to the variable. Most variable overrides modify the values of flags or the names of commands executed in recipes. For example, the default value of CFLAGS is often overridden or augmented and the ANSI/POSIX Computing Environment is selected by setting the CC and LD variables to pcc.
Modifying rules is trickier than modifying variables. Additional constraints can be added to a rule by specifying the target and the new prerequisite. For example,
%.$O: header.h
adds file header.h the set of prerequisites for all object files. There is no mechanism for adding additional commands to an existing recipe; if a recipe is unsatisfactory, the rule and its recipe must be completely overridden. A rule is overridden only when the replacement rule matches the target and prerequisite portions of the original rule exactly. The recipe associated with the new rule then replaces the recipe of the original rule. For example, /sys/src/cmd/lex/mkfile overrides the default installall rule to perform the normal loop on all architectures and then copy a prototype file to the system library directory.
</$objtype/mkfile
TARG=lex
OFILES=lmain.$O\
y.tab.$O\
sub1.$O\
sub2.$O\
header.$O\
HFILES=ldefs.h\
YFILES=parser.y\
BIN=/$objtype/bin
</sys/src/cmd/mkone
installall:V:
for(objtype in $CPUS)
mk install
cp ncform /sys/lib/lex
Another way to perform the same override is to add a dependency to the default installall rule that executes an additional rule to install the prototype file:
installall:V: ncform.install
ncform.install:V:
cp ncform /sys/lib/lex
Special Tricks
Two special cases require extra deviousness.
In the first, a file needed to build an executable is generated by a program that, in turn, is built from a source file that is not part of the product. In this case, the executable must be built for the target architecture, but the intermediate executable must be built for the architecture mk is executing on. The intermediate executable is built by recursively invoking mk with the appropriate target and the executing architecture as the target architecture. When that mk completes, the intermediate is executed to generate the source file to complete the build for the target architecture. The earlier example of /sys/src/cmd/awk/mkfile illustrates this technique.
Another awkward situation occurs when a directory contains source to build an executable as well as source for auxiliary executables that are not to be installed. In this case the mkmany generic rules are inappropriate, because all executables would be built and installed. Instead, use the mkone generic file to build the primary executable and provide extra targets to build the auxiliary files. This approach is also useful when the auxiliary files are not executables; /sys/src/cmd/spell/mkfile augments the default rules to build and install the spell executable with elaborate rules to generate and maintain the auxiliary spelling lists.