GNU Make in Detail for Beginners

Large projects can contain thousands of lines of code, distributed in multiple source files, written by many developers and arranged in several subdirectories. A project may contain several component divisions. These components may have complex inter-dependencies
— for example, in order to compile component X, you have to first compile Y; in order to compile Y, you have to first compile Z; and so on. For a large project, when a few changes are made to the source, manually recompiling the entire project each time is
tedious, error-prone and time-consuming.

Make is a solution to these problems. It can be used to specify dependencies between components, so that it will compile components in the order required to satisfy dependencies. An important feature is that when a project is recompiled after a few changes,
it will recompile only the files which are changed, and any components that are dependent on it. This saves a lot of time. Make is, therefore, an essential tool for a large software project.

Each project needs a Makefile — a script that describes the project structure, namely, the source code files, the dependencies between them, compiler arguments, and how to produce the target output (normally, one or more executables). Whenever the make command
is executed, the Makefile in the current working directory is interpreted, and the instructions executed to produce the target outputs. The Makefile contains a collection of rules, macros, variable assignments, etc. (‘Makefile’ or ‘makefile’ are both acceptable.)

Installing GNU Make

Most distributions don’t ship make as part of the default installation. You have to install it, either using the package-management system, or by manually compiling from source. To compile and build from source, download
the tarball, extract it, and go through the README file. (If you’re running Ubuntu, you can install make as well as some other common packages required for building from source, by running:

sudo
apt-get install build-essential

.)

A sample project

To acquaint ourselves with the basics of make, let’s use a simple C “Hello world” project, and a Makefile that handles building of the target binary. We have three files (below):

module.h

,
the header file that contains the declarations;

module.c

, which contains the definition of
the function defined in

module.h

; and the main file,

main.c

,
in which we call the

sample_func()

defined in

module.c

.
Since

module.h

includes the required header files like

stdio.h

,
we don’t need to include

stdio.h

in every module; instead, we just include

module.h

.
Here,

module.c

and

main.c

can
be compiled as separate object modules, and can be linked by GCC to obtain the target binary.

module.h:

#include <stdio.h>
voidsample_func();

module.c:

#include "module.h"
voidsample_func()
{
printf("Hello world!");
}

main.c:

#include "module.h"
voidsample_func();
intmain()
{
sample_func();
return0;
}

The following are the manual steps to compile the project and produce the target binary:

slynux@freedom:~$ gcc -I . -c main.c # Obtain main.o
slynux@freedom:~$ gcc -I . -c module.c # Obtain module.o
slynux@freedom:~$ gcc main.o module.o -o target_bin #Obtain target binary

(

-I

is used to include the current directory (

.

)
as a header file location.)

Writing a Makefile from scratch

By convention, all variable names used in a Makefile are in upper-case. A common variable assignment in a Makefile is

CC
= gcc

, which can then be used later on as

${CC}

or

$(CC)

.
Makefiles use

#

as the comment-start marker, just like in shell scripts.

The general syntax of a Makefile rule is as follows:

target: dependency1 dependency2 ...
[TAB] action1
[TAB] action2
...

Let’s take a look at a simple Makefile for our sample project:

all: main.o module.o
gcc main.o module.o -o target_bin
main.o: main.c module.h
gcc -I . -c main.c
module.o: module.c module.h
gcc -I . -c module.c
clean:
rm -rf *.o
rm target_bin

We have four targets in the Makefile:

all

is a special target that depends on

main.o

and

module.o

,
and has the command (from the “manual” steps earlier) to make GCC link the two object files into the final executable binary.

main.o

is a filename target that depends on

main.c

and

module.h

,
and has the command to compile

main.c

to produce

main.o

.

module.o

is a filename target that depends on

module.c

and

module.h

;
it calls GCC to compile the

module.c

file to produce

module.o

.

clean

is a special target that has no dependencies, but specifies the commands to clean the
compilation outputs from the project directories.

You may be wondering why the order of the make targets and commands in the Makefile are not the same as that of the manual compilation commands we ran earlier. The reason is so that the easiest invocation, by just calling the make command, will result in the
most commonly desired output — the final executable. How does this work?

The make command accepts a target parameter (one of those defined in the Makefile), so the generic command line syntax is

make
<target>

. However, make also works if you do not specify any target on the command line, saving you a little typing; in such a case, it defaults to the first target defined in the Makefile. In our Makefile, that is the target

all

,
which results in the creation of the desired executable binary

target_bin

!

Makefile processing, in general

When the make command is executed, it looks for a file named

makefile

or

Makefile

in
the current directory. It parses the found

Makefile

, and constructs a dependency tree. Based
on the desired make target specified (or implied) on the command-line, make checks if the dependency files of that target exist. And (for filename targets — explained below) if they exist, whether they are newer than the target itself, by comparing file timestamps.

Before executing the action (commands) corresponding to the desired target, its dependencies must be met; when they are not met, the targets corresponding to the unmet dependencies are executed before the given make target, to supply the missing dependencies.

When a target is a filename, make compares the timestamps of the target file and its dependency files. If the dependency filename is another target in the Makefile, make then checks the timestamps of that target’s dependencies. It thus winds up recursively
checking all the way down the dependency tree, to the source code files, to see if any of the files in the dependency tree are newer than their target filenames. (Of course, if the dependency files don’t exist, then make knows it must start executing the make
targets from the “lowest” point in the dependency tree, to create them.)

If make finds that files in the dependency tree are newer than their target, then all the targets in the affected branch of the tree are executed, starting from the “lowest”, to update the dependency files. When make finally returns from its recursive checking
of the tree, it completes the final comparison for the desired make target. If the dependency files are newer than the target (which is usually the case), it runs the command(s) for the desired make target.

This process is how make saves time, by executing only commands that need to be executed, based on which of the source files (listed as dependencies) have been updated, and have a newer timestamp than their target.

Now, when a target is not a filename (like all and clean in our Makefile, which we called “special targets”), make obviously cannot compare timestamps to check whether the target’s dependencies are newer. Therefore, such a target is always executed, if specified
(or implied) on the command line.

For the execution of each target, make prints the actions while executing them. Note that each of the actions (shell commands written on a line) are executed in a separate sub-shell. If an action changes the shell environment, such a change is restricted to
the sub-shell for that action line only. For example, if one action line contains a command like

cd
newdir

, the current directory will be changed only for that line/action; for the next line/action, the current directory will be unchanged.

Processing our Makefile

After understanding how make processes Makefiles, let’s run make on our own Makefile, and see how it is processed to illustrate how it works. In the project directory, we run the following command:

slynux@freedom:~$ make
gcc -I . -c main.c
gcc -I . -c module.c
gcc main.o module.o -o target_bin

What has happened here?

When we ran make without specifying a target on the command line, it defaulted to the first target in our Makefile — that is, the target

all

.
This target’s dependencies are

module.o

and

main.o

.
Since these files do not exist on our first run of make for this project, make notes that it must execute the targets

main.o

and

module.o

.
These targets, in turn, produce the

main.o

and

module.o

files
by executing the corresponding actions/commands. Finally, make executes the command for the target

all

.
Thus, we obtain our desired output,

target_bin

.

If we immediately run make again, without changing any of the source files, we will see that only the command for the target

all

is
executed:

slynux@freedom:~$ make
gcc main.o module.o -o target_bin

Though make checked the dependency tree, neither of the dependency targets (

module.o

and

main.o

)
had their own dependency files bearing a later timestamp than the dependency target filename. Therefore, make rightly did not execute the commands for the dependency targets. As we mentioned earlier, since the target

all

is
not a filename, make cannot compare file timestamps, and thus executes the action/command for this target.

Now, we update

module.c

by adding a statement

printf("\nfirst
update");

inside the

sample_func()

function. We then run make again:

slynux@freedom:~$ make
gcc -I . -c module.c
gcc main.o module.o -o target_bin

Since

module.c

in the dependency tree has changed (it now has a later timestamp than its target,

module.o

),
make runs the action for the

module.o

target, which recompiles the changed source file. It
then runs the action for the all target.

We can explicitly invoke the

clean

target to clean up all the generated

.o

files
and

target_bin

:

$ make clean
rm -rf *.o
rm target_bin

More bytes on Makefiles

Make provides many interesting features that we can use in Makefiles. Let’s look at the most essential ones.

Dealing with assignments

There are different ways of assigning variables in a Makefile. They are (type of assignment, followed by the operator in parentheses):

Simple assignment (:=)

We can assign values (RHS) to variables (LHS) with this operator, for example:

CC := gcc

.
With simple assignment (

:=

), the value is expanded and stored to all occurrences in the Makefile
when its first definition is found.

For example, when a

CC := ${GCC} ${FLAGS}

simple definition is first encountered,

CC

is
set to

gcc -W

and wherever

${CC}

occurs
in actions, it is replaced with

gcc -W

.

Recursive assignment (=)

Recursive assignment (the operator used is

=

) involves variables and values that are not evaluated
immediately on encountering their definition, but are re-evaluated every time they are encountered in an action that is being executed. As an example, say we have:

GCC = gcc
FLAGS = -W

With the above lines,

CC = ${GCC} {FLAGS}

will be converted to

gcc
-W

only when an action like

${CC} file.c

is executed somewhere in the Makefile. With
recursive assignation, if the GCC variable is changed later (for example,

GCC = c++

), then
when it is next encountered in an action line that is being updated, it will be re-evaluated, and the new value will be used;

${CC}

will
now expand to

c++ -W

.

We will also have an interesting and useful application further in the article, where this feature is used to deal with varying cases of filename extensions of image files.

Conditional assignment (?=)

Conditional assignment statements assign the given value to the variable only if the variable does not yet have a value.

Appending (+=)

The appending operation appends texts to an existing variable. For example:

CC = gcc
CC += -W

CC

now holds the value

gcc
-W

.

Though variable assignments can occur in any part of the Makefile, on a new line, most variable declarations are found at the beginning of the Makefile.

Using patterns and special variables

The

%

character can be used for wildcard pattern-matching, to provide generic targets. For
example:

%.o: %.c
[TAB] actions

When

%

appears in the dependency list, it is replaced with the same string that was used to
perform substitution in the target.

Inside actions, we can use special variables for matching filenames. Some of them are:

$@

(full target name of the current target)

$?

(returns the dependencies that are newer than the current target)

$*

(returns the text that corresponds to

%

in
the target)

$<

(name of the first dependency)

$^

(name of all the dependencies with space as the delimiter)

Instead of writing each of the file names in the actions and the target, we can use shorthand notations based on the above, to write more generic Makefiles.

Action modifiers

We can change the behaviour of the actions we use by prefixing certain action modifiers to the actions. Two important action modifiers are:

-

(minus) — Prefixing this to any action causes any error that occurs while executing the
action to be ignored. By default, execution of a Makefile stops when any command returns a non-zero (error) value. If an error occurs, a message is printed, with the status code of the command, and noting that the error has been ignored. Looking at the Makefile
from our sample project: in the clean target, the

rm target_bin

command will produce an error
if that file does not exist (this could happen if the project had never been compiled, or if

make
clean

is run twice consecutively). To handle this, we can prefix the

rm

command with
a minus, to ignore errors:

-rm target_bin

.

@

(at) suppresses the standard print-action-to-standard-output behaviour of make, for the
action/command that is prefixed with

@

. For example, to echo a custom message to standard
output, we want only the output of the echo command, and don’t want to print the echo command line itself.

@echo
Message

will print “Message” without the echo command line being printed.

Use PHONY to avoid file-target name conflicts

Remember the

all

and

clean

special
targets in our Makefile? What happens when the project directory has files with the names

all

or

clean

?
The conflicts will cause errors. Use the

.PHONY

directive to specify which targets are not
to be treated as files — for example:

.PHONY: all clean

.

Simulating make without actual execution

At times, maybe when developing the Makefile, we may want to trace the make execution (and view the logged messages) without actually running the actions, which is time consuming. Simply use

make
-n

to do a “dry run”.

Using the shell command output in a variable

Sometimes we need to use the output from one command/action in other places in the Makefile — for example, checking versions/locations of installed libraries, or other files required for compilation. We can obtain the shell output using the shell command. For
example, to return a list of files in the current directory into a variable, we would run:

LS_OUT
= $(shell ls)

.

Nested Makefiles

Nested Makefiles (which are Makefiles in one or more subdirectories that are also executed by running the make command in the parent directory) can be useful for building smaller projects as part of a larger project. To do this, we set up a target whose action
changes directory to the subdirectory, and invokes make again:

subtargets:
cd subdirectory && $(MAKE)

Instead of running the make command, we used

$(MAKE)

, an environment variable, to provide
flexibility to include arguments. For example, if you were doing a “dry run” invocation: if we used the make command directly for the subdirectory, the simulation option (

-n

)
would not be passed, and the commands in the subdirectory’s Makefile would actually be executed. To enable use of the

-n

argument,
use the

$(MAKE)

variable.

Now let’s improve our original Makefile using these advanced features:

CC = gcc # Compiler to use
OPTIONS = -O2 -g -Wall # -g for debug, -O2 for optimise and -Wall additional messages
INCLUDES = -I . # Directory for header file
OBJS = main.o module.o # List of objects to be build
.PHONY: all clean # To declare all, clean are not files

all: ${OBJS}
@echo "Building.." # To print "Building.." message
${CC} ${OPTIONS} ${INCLUDES} ${OBJS} -o target_bin

%.o: %.c  # % pattern wildcard matching
${CC} ${OPTIONS} -c $*.c ${INCLUDES}
list:
@echo $(shell ls) # To print output of command 'ls'

clean:
@echo "Cleaning up.."
-rm -rf *.o # - prefix for ignoring errors and continue execution
-rm target_bin

Run make on the modified Makefile and test it; also run make with the new list target. Observe the output.

Make in non-compilation contexts

I hope you’re now well informed about using make in a programming context. However, it’s also useful in non-programming contexts, due to the basic behaviour of checking the modification timestamps of target files and dependencies, and running the specified
actions when required. For example, let’s write a Makefile that will manage an image store for us, doing thumbnailing when required. Our scenario is as follows:

We have a directory with two subdirectories,

images

and

thumb

.

The

images

subdirectory contains many large image files;

thumb

contains
thumbnails of the images, as

.jpg

files, 100x100px in image size.

When a new image is added to the images directory, creation of its thumbnail in the

thumb

directory
should be automated. If an image is modified, its thumbnail should be updated.

The thumbnailing process should only be done for new or updated images, and not images that have up-to-date thumbnails.

This problem can be solved easily by creating a Makefile in the top-level directory, as follows:

FILES = $(shell  find images -type f -iname "*.jpg" | sed 's/images/thumb/g')
CONVERT_CMD = convert -resize "100x100" $< $@
MSG = @echo "\nUpdating thumbnail" $@

all: ${FILES}
thumb/%.jpg: images/%.jpg
$(MSG)
$(CONVERT_CMD)
thumb/%.JPG: images/%.JPG
$(MSG)
$(CONVERT_CMD)
clean:
@echo Cleaning up files..
rm -rf thumb/*.jpg thumb/*.JPG

In the above Makefile,

FILES = $(shell  find images -type f -iname "*.jpg" | sed 's/images/thumb/g')

is
used to generate a list of dependency filenames. JPEG files could have the extension

.jpg

or

.JPG

(that
is, differing in case). The

-iname

parameter to find (

find
images -type f -iname "*.jpg"

) will do a case-insensitive search on the names of files, and will return files with both lower-case and upper-case extensions — for example,

images/1.jpg

,

images/2.jpg

,

images/3.JPG

and
so on. The

sed

command replaces the text “images” with “thumb”, to get the dependency file
path.

When make is invoked, the all target is executed first. Since FILES contains a list of thumbnail files for which to check the timestamp (or if they exist), make jumps down to the

thumb/%.jpg

wildcard
target for each thumbnail image file name. (If the extension is upper-case, that is,

thumb/3.JPG

,
then make will look for, and find, the second wildcard target,

thumb/%.JPG

.)

For each thumbnail file in the

thumb

directory, its dependency is the image file in the

images

directory.
Hence, if any file (that’s expected to be) in the

thumb

directory does not exist, or its
timestamp is older than the dependency file in the

images

directory, the action (calling

$(CONVERT_CMD

)
to create a thumbnail) is run.

Using the features we described earlier,

CONVERT_CMD

is defined before targets are specified,
but it uses recursive assignment. Hence, the input and target filenames passed to the convert command are substituted from the first dependency (

$<

)
and the target (

$@

) every time the action is invoked, and thus will work no matter from which
action target (

thumb/%.JPG

or

thumb/%.jpg

)
the action is invoked.

Naturally, the “Updating thumbnail” message is also defined using recursive assignment for the same reasons, ensuring that

$(MSG)

is
re-evaluated every time the actions are executed, and thereby able to cope with variations in the case of the filename extension.

slynux@freedom:~$ make
Updating thumbnail 1.jpg
convert -resize "100x100" images/1.jpg thumb/1.jpg
… …Updating thumbnail 4.jpg
convert -resize "100x100" images/4.jpg thumb/4.jpg

If I edit

4.jpg

in images and rerun make, since only

4.jpg

‘s
timestamp has changed, a thumbnail is generated for that image:

slynux@freedom:~$ make
Updating thumbnail 4.jpg
convert -resize "100x100" images/4.jpg thumb/4.jpg

Writing a script (shell script or Python, etc) to maintain image thumbnails by monitoring timestamps would have taken many lines of code. With make, we can do this in just 8 lines of Makefile. Isn’t make awesome?

from http://www.linuxforu.com/2012/06/gnu-make-in-detail-for-beginners/