present in all implementations, and have been marked obsolete by Posix
2008. This is because there are inherent ambiguities in using them.
For example, ‘test "$1" -a "$2"’ looks like a binary operator to
check whether two strings are both non-empty, but if ‘$1’ is the
literal ‘!’, then some implementations of test
treat it
as a negation of the unary operator -a.
Thus, portable uses of test
should never have more than four
arguments, and scripts should use shell constructs like ‘&&’ and
‘||’ instead. If you combine ‘&&’ and ‘||’ in the same
statement, keep in mind that they have equal precedence, so it is often
better to parenthesize even when this is redundant. For example:
# Not portable: test "X$a" = "X$b" -a \ '(' "X$c" != "X$d" -o "X$e" = "X$f" ')' # Portable: test "X$a" = "X$b" && { test "X$c" != "X$d" || test "X$e" = "X$f"; }
test
does not process options like most other commands do; for
example, it does not recognize the -- argument as marking the
end of options.
It is safe to use ‘!’ as a test
operator. For example,
‘if test ! -d foo; …’ is portable even though ‘if ! test
-d foo; …’ is not.
test
(files)To enable configure
scripts to support cross-compilation, they
shouldn’t do anything that tests features of the build system instead of
the host system. But occasionally you may find it necessary to check
whether some arbitrary file exists. To do so, use ‘test -f’,
‘test -r’, or ‘test -x’. Do not use ‘test -e’, because
Solaris 10 /bin/sh
lacks it. To test for symbolic links on systems that have them, use
‘test -h’ rather than ‘test -L’; either form conforms to
Posix 1003.1-2001, but older shells like Solaris 8
/bin/sh
support only -h.
For historical reasons, Posix reluctantly allows implementations of
‘test -x’ that will succeed for the root user, even if no execute
permissions are present. Furthermore, shells do not all agree on
whether Access Control Lists should affect ‘test -r’, ‘test
-w’, and ‘test -x’; some shells base test results strictly on the
current user id compared to file owner and mode, as if by
stat(2)
; while other shells base test results on whether the
current user has the given right, even if that right is only granted by
an ACL, as if by faccessat(2)
. Furthermore, there is a classic
time of check to time of use race between any use of test
followed by operating on the just-checked file. Therefore, it is a good
idea to write scripts that actually attempt an operation, and are
prepared for the resulting failure if permission is denied, rather than
trying to avoid an operation based solely on whether test
guessed that it might not be permitted.
test
(strings)Posix says that ‘test "string"’ succeeds if string is
not null, but this usage is not portable to traditional platforms like
Solaris 10 /bin/sh
, which mishandle strings like ‘!’ and
‘-n’. However, it is portable to test if a variable is set
to a non-empty value, by using ‘test ${var+y}’, since all known
implementations properly distinguish between no arguments and a
known-safe string of ‘y’.
Posix also says that ‘test ! "string"’, ‘test -n "string"’ and ‘test -z "string"’ work with any string, but many shells (such as Solaris 10, AIX 3.2, UNICOS <IP_ADDRESS>, Digital Unix 4, etc.) get confused if string looks like an operator:
$ test -n = test: argument expected $ test ! -n test: argument expected $ test -z ")"; echo $? 0
Similarly, Posix says that both ‘test "string1" = "string2"’ and ‘test "string1" != "string2"’ work for any pairs of strings, but in practice this is not true for troublesome strings that look like operators or parentheses, or that begin with ‘-’.
It is best to protect such strings with a leading ‘X’, e.g., ‘test "Xstring" != X’ rather than ‘test -n "string"’ or ‘test ! "string"’.
It is common to find variations of the following idiom:
test -n "`echo $ac_feature | sed 's/[-a-zA-Z0-9_]//g'`" && action
to take an action when a token matches a given pattern. Such constructs should be avoided by using:
case $ac_feature in *[!-a-zA-Z0-9_]*) action;; esac
If the pattern is a complicated regular expression that cannot be expressed as a shell pattern, use something like this instead:
expr "X$ac_feature" : 'X.*[^-a-zA-Z0-9_]' >/dev/null && action
‘expr "Xfoo" : "Xbar"’ is more robust than ‘echo "Xfoo" | grep "^Xbar"’, because it avoids problems when ‘foo’ contains backslashes.
trap
It is safe to trap at least the signals 1, 2, 13, and 15. You can also
trap 0, i.e., have the trap
run when the script ends (either via an
explicit exit
, or the end of the script). The trap for 0 should be
installed outside of a shell function, or AIX 5.3 /bin/sh
will invoke the trap at the end of this function.
Posix says that ‘trap - 1 2 13 15’ resets the traps for the
specified signals to their default values, but many common shells (e.g.,
Solaris 10 /bin/sh
) misinterpret this and attempt to execute a
“command” named -
when the specified conditions arise.
Posix 2008 also added a requirement to support ‘trap 1 2 13 15’ to
reset traps, as this is supported by a larger set of shells, but there
are still shells like dash
that mistakenly try to execute
1
instead of resetting the traps. Therefore, there is no
portable workaround, except for ‘trap - 0’, for which
‘trap '' 0’ is a portable substitute.
Although Posix is not absolutely clear on this point, it is widely
admitted that when entering the trap ‘$?’ should be set to the exit
status of the last command run before the trap. The ambiguity can be
summarized as: “when the trap is launched by an exit
, what is
the last command run: that before exit
, or
exit
itself?”
Bash considers exit
to be the last command, while Zsh and
Solaris 10 /bin/sh
consider that when the trap is run it is
still in the exit
, hence it is the previous exit status
that the trap receives:
$ cat trap.sh trap 'echo $?' 0 (exit 42); exit 0 $ zsh trap.sh 42 $ bash trap.sh 0
The portable solution is then simple: when you want to ‘exit 42’,
run ‘(exit 42); exit 42’, the first exit
being used to
set the exit status to 42 for Zsh, and the second to trigger the trap
and pass 42 as exit status for Bash. In M4sh, this is covered by using
AS_EXIT
.
The shell in FreeBSD 4.0 has the following bug: ‘$?’ is
reset to 0 by empty lines if the code is inside trap
.
$ trap 'false echo $?' 0 $ exit 0
Fortunately, this bug only affects trap
.
Several shells fail to execute an exit trap that is defined inside a subshell, when the last command of that subshell is not a builtin. A workaround is to use ‘exit $?’ as the shell builtin.
$ bash -c '(trap "echo hi" 0; /bin/true)' hi $ /bin/sh -c '(trap "echo hi" 0; /bin/true)' $ /bin/sh -c '(trap "echo hi" 0; /bin/true; exit $?)' hi
Likewise, older implementations of bash
failed to preserve
‘$?’ across an exit trap consisting of a single cleanup command.
$ bash -c 'trap "/bin/true" 0; exit 2'; echo $? 2 $ bash-2.05b -c 'trap "/bin/true" 0; exit 2'; echo $? 0 $ bash-2.05b -c 'trap ":; /bin/true" 0; exit 2'; echo $? 2
Be aware that a trap can be called from any number of places in your
script, and therefore the trap handler should not make assumptions about
shell state. For some examples, if your script temporarily modifies
IFS
, then the trap should include an initialization back to its
typical value of space-tab-newline (autoconf does this for generated
configure files). Likewise, if your script changes the current
working directory at some point after the trap is installed, then your
trap cannot assume which directory it is in, and should begin by
changing directories to an absolute path if that is important to the
cleanup efforts (autotest does this for generated testsuite
files).
true
Don’t worry: as far as we know true
is portable.
Nevertheless, it’s not always a builtin (e.g., Bash 1.x), and the
portable shell community tends to prefer using :
. This has a
funny side effect: when asked whether false
is more portable
than true
Alexandre Oliva answered:
In a sense, yes, because if it doesn’t exist, the shell will produce an exit status of failure, which is correct for
false
, but not fortrue
.
Remember that even though ‘:’ ignores its arguments, it still takes time to compute those arguments. It is a good idea to use double quotes around any arguments to ‘:’ to avoid time spent in field splitting and file name expansion.
unset
In some nonconforming shells (e.g., Solaris 10 /bin/ksh
and
/usr/xpg4/bin/sh
, NetBSD 5.99.43 sh, or Bash 2.05a),
unset FOO
fails when FOO
is not set. This can interfere
with set -e
operation. You can use
FOO=; unset FOO
if you are not sure that FOO
is set.
A few ancient shells lack unset
entirely. For some variables
such as PS1
, you can use a neutralizing value instead:
PS1='$ '
Usually, shells that do not support unset
need less effort to
make the environment sane, so for example is not a problem if you cannot
unset CDPATH
on those shells. However, Bash 2.01 mishandles
unset MAIL
and unset MAILPATH
in some cases and dumps core.
So, you should do something like
( (unset MAIL) || exit 1) >/dev/null 2>&1 && unset MAIL || :
See Special Shell Variables, for some neutralizing values. Also, see Limitations of Builtins, for the case of environment variables.
wait
The exit status of wait
is not always reliable.
Previous: Limitations of Shell Builtins, Up: Portable Shell Programming [Contents][Index]
The small set of tools you can expect to find on any machine can still include some limitations you should be aware of.
awk
Don’t leave white space before the opening parenthesis in a user function call. Posix does not allow this and GNU Awk rejects it:
$ gawk 'function die () { print "Aaaaarg!"  } BEGIN { die() }' Aaaaarg!
Posix says that if a program contains only ‘BEGIN’ actions, and
contains no instances of getline
, then the program merely
executes the actions without reading input. However, traditional Awk
implementations (such as Solaris 10 awk
) read and discard
input in this case. Portable scripts can redirect input from
/dev/null to work around the problem. For example:
awk 'BEGIN {print "hello world"}' </dev/null
Posix says that in an ‘END’ action, ‘$NF’ (and presumably, ‘$1’) retain their value from the last record read, if no intervening ‘getline’ occurred. However, some implementations (such as Solaris 10 ‘/usr/bin/awk’, ‘nawk’, or Darwin ‘awk’) reset these variables. A workaround is to use an intermediate variable prior to the ‘END’ block. For example:
$ cat end.awk { tmp = $1 } END { print "a", $1, $NF, "b", tmp } $ echo 1 | awk -f end.awk a b 1 $ echo 1 | gawk -f end.awk a 1 1 b 1
If you want your program to be deterministic, don’t depend on for
on arrays:
$ cat for.awk END { arr["foo"] = 1 arr["bar"] = 1 for (i in arr) print i } $ gawk -f for.awk </dev/null foo bar $ nawk -f for.awk </dev/null bar foo
Some Awk implementations, such as HP-UX 11.0’s native one, mishandle anchors:
$ echo xfoo | $AWK '/foo|^bar/ { print }' $ echo bar | $AWK '/foo|^bar/ { print }' bar $ echo xfoo | $AWK '/^bar|foo/ { print }' xfoo $ echo bar | $AWK '/^bar|foo/ { print }' bar
Either do not depend on such patterns (i.e., use ‘/^(.*foo|bar)/’, or use a simple test to reject such implementations.
On ‘ia64-hp-hpux11.23’, Awk mishandles printf
conversions
after %u
:
$ awk 'BEGIN { printf "%u %d\n", 0, -1 }' 0 0
AIX version 5.2 has an arbitrary limit of 399 on the length of regular expressions and literal strings in an Awk program.
Traditional Awk implementations derived from Unix version 7, such as
Solaris /bin/awk
, have many limitations and do not
conform to Posix. Nowadays AC_PROG_AWK
(see Particular Program Checks) finds you an Awk that doesn’t have these problems, but if
for some reason you prefer not to use AC_PROG_AWK
you may need to
address them. For more detailed descriptions, see awk
language history in GNU Awk User’s Guide.
Traditional Awk does not support multidimensional arrays or user-defined functions.
Traditional Awk does not support the -v option. You can use
assignments after the program instead, e.g., $AWK '{print v
$1}' v=x
; however, don’t forget that such assignments are not
evaluated until they are encountered (e.g., after any BEGIN
action).
Traditional Awk does not support the keywords delete
or do
.
Traditional Awk does not support the expressions
a?b:c
, !a
, a^b
,
or a^=b
.
Traditional Awk does not support the predefined CONVFMT
or
ENVIRON
variables.
Traditional Awk supports only the predefined functions exp
, index
,
int
, length
, log
, split
, sprintf
,
sqrt
, and substr
.
Traditional Awk getline
is not at all compatible with Posix;
avoid it.
Traditional Awk has for (i in a) …
but no other uses of the
in
keyword. For example, it lacks if (i in a) …
.
In code portable to both traditional and modern Awk, FS
must be a
string containing just one ordinary character, and similarly for the
field-separator argument to split
.
Traditional Awk has a limit of 99 fields in a record. Since some Awk
implementations, like Tru64’s, split the input even if you don’t refer
to any field in the script, to circumvent this problem, set ‘FS’
to an unusual character and use split
.
Traditional Awk has a limit of at most 99 bytes in a number formatted by
OFMT
; for example, OFMT="%.300e"; print 0.1;
typically
dumps core.
The original version of Awk had a limit of at most 99 bytes per
split
field, 99 bytes per substr
substring, and 99 bytes
per run of non-special characters in a printf
format, but these
bugs have been fixed on all practical hosts that we know of.
HP-UX 11.00 and IRIX 6.5 Awk require that input files have a line length of at most 3070 bytes.
basename
Not all hosts have a working basename
.
You can use expr
instead.
cat
Don’t rely on any option.
cc
The command ‘cc -c foo.c’ traditionally produces an object file
named foo.o. Most compilers allow -c to be combined
with -o to specify a different object file name, but
Posix does not require this combination and a few compilers
lack support for it. See C Compiler Characteristics, for how GNU Make
tests for this feature with AC_PROG_CC_C_O
.
When a compilation such as ‘cc -o foo foo.c’ fails, some compilers (such as CDS on Reliant Unix) leave a foo.o.
HP-UX cc
doesn’t accept .S files to preprocess and
assemble. ‘cc -c foo.S’ appears to succeed, but in fact does
nothing.
The default executable, produced by ‘cc foo.c’, can be
gcc
).
gcc
.
cc
wrapper for DEC C on OpenVMS.
The C compiler’s traditional name is cc
, but other names like
gcc
are common. Posix 1003.1-2001 and 1003.1-2008 specify the
name c99
, but older Posix editions specified
c89
, future POSIX standards will likely specify
c11
, and anyway these standard names are rarely used in
practice. Typically the C compiler is invoked from makefiles that use
‘$(CC)’, so the value of the ‘CC’ make variable selects the
compiler name.
chgrp
chown
It is not portable to change a file’s group to a group that the owner does not belong to.
chmod
Avoid usages like ‘chmod -w file’; use ‘chmod a-w file’ instead, for two reasons. First, plain -w does not necessarily make the file unwritable, since it does not affect mode bits that correspond to bits in the file mode creation mask. Second, Posix says that the -w might be interpreted as an implementation-specific option, not as a mode; Posix suggests using ‘chmod -- -w file’ to avoid this confusion, but unfortunately ‘--’ does not work on some older hosts.
cmp
cmp
performs a raw data comparison of two files, while
diff
compares two text files. Therefore, if you might compare
DOS files, even if only checking whether two files are different, use
diff
to avoid spurious differences due to differences of
newline encoding.
cp
Avoid the -r option, since Posix 1003.1-2004 marks it as
obsolescent and its behavior on special files is implementation-defined.
Use -R instead. On GNU hosts the two options
are equivalent, but on Solaris hosts (for example) cp -r
reads from pipes instead of replicating them. AIX 5.3 cp -R
may
corrupt its own memory with some directory hierarchies and error out or
dump core:
mkdir -p 12345678/12345678/12345678/12345678 touch 12345678/12345678/x cp -R 12345678 t cp: 0653-440 12345678/12345678/: name too long.
Some cp
implementations (e.g., BSD/OS 4.2) do not allow
trailing slashes at the end of nonexistent destination directories. To
avoid this problem, omit the trailing slashes. For example, use
‘cp -R source /tmp/newdir’ rather than ‘cp -R source
/tmp/newdir/’ if /tmp/newdir does not exist.
The -f option is portable nowadays.
Traditionally, file timestamps had 1-second resolution, and ‘cp
-p’ copied the timestamps exactly. However, many modern file systems
have timestamps with 1-nanosecond resolution. Unfortunately, some older
‘cp -p’ implementations truncate timestamps when copying files,
which can cause the destination file to appear to be older than the
source. The exact amount of truncation depends on the resolution of
the system calls that cp
uses. Traditionally this was
utime
, which has 1-second resolution. Less-ancient cp
implementations such as GNU Core Utilities 5.0.91 (2003) use
utimes
, which has 1-microsecond resolution. Modern
implementations such as GNU Core Utilities 6.12 (2008) can set timestamps to
the full nanosecond resolution, using the modern system calls
futimens
and utimensat
when they are available. As of
2011, though, many platforms do not yet fully support these new system
calls.
Bob Proulx notes that ‘cp -p’ always tries to copy
ownerships. But whether it actually does copy ownerships or not is a
system dependent policy decision implemented by the kernel. If the
kernel allows it then it happens. If the kernel does not allow it then
it does not happen. It is not something cp
itself has control
over.
In Unix System V any user can chown files to any other user, and System
V also has a non-sticky /tmp. That probably derives from the
heritage of System V in a business environment without hostile users.
BSD changed this
to be a more secure model where only root can chown
files and
a sticky /tmp is used. That undoubtedly derives from the heritage
of BSD in a campus environment.
GNU/Linux and Solaris by default follow BSD, but
can be configured to allow a System V style chown
. On the
other hand, HP-UX follows System V, but can
be configured to use the modern security model and disallow
chown
. Since it is an administrator-configurable parameter
you can’t use the name of the kernel as an indicator of the behavior.
date
Some versions of date
do not recognize special ‘%’ directives,
and unfortunately, instead of complaining, they just pass them through,
and exit with success:
$ uname -a OSF1 medusa.sis.pasteur.fr V5.1 732 alpha $ date "+%s" %s
diff
Option -u is nonportable.
Some implementations, such as Tru64’s, fail when comparing to /dev/null. Use an empty file instead.
dirname
Not all hosts have a working dirname
, and you should instead
use AS_DIRNAME
(see Programming in M4sh). For example:
dir=`dirname "$file"` # This is not portable. dir=`AS_DIRNAME(["$file"])` # This is more portable.
egrep
Although Posix stopped requiring egrep
in 2001,
a few traditional hosts (notably Solaris) do not support the Posix
replacement grep -E
. Also, some traditional implementations do
not work on long input lines. To work around these problems, invoke
AC_PROG_EGREP
and then use $EGREP
.
Portable extended regular expressions should use ‘\’ only to escape characters in the string ‘$()*+.?[\^{|’. For example, ‘\}’ is not portable, even though it typically matches ‘}’.
The empty alternative is not portable. Use ‘?’ instead. For instance with Digital Unix v5.0:
> printf "foo\n|foo\n" | $EGREP '^(|foo|bar)$' |foo > printf "bar\nbar|\n" | $EGREP '^(foo|bar|)$' bar| > printf "foo\nfoo|\n|bar\nbar\n" | $EGREP '^(foo||bar)$' foo |bar
$EGREP
also suffers the limitations of grep
(see Limitations of Usual Tools).
expr
Not all implementations obey the Posix rule that ‘--’ separates
options from arguments; likewise, not all implementations provide the
extension to Posix that the first argument can be treated as part of a
valid expression rather than an invalid option if it begins with
‘-’. When performing arithmetic, use ‘expr 0 + $var’ if
‘$var’ might be a negative number, to keep expr
from
interpreting it as an option.
No expr
keyword starts with ‘X’, so use ‘expr
X"word" : 'Xregex'’ to keep expr
from
misinterpreting word.
Don’t use length
, substr
, match
and index
.
expr
(‘|’) ¶You can use ‘|’. Although Posix does require that ‘expr ''’ return the empty string, it does not specify the result when you ‘|’ together the empty string (or zero) with the empty string. For example:
expr '' \| ''
Posix 1003.2-1992 returns the empty string for this case, but traditional Unix returns ‘0’ (Solaris is one such example). In Posix 1003.1-2001, the specification was changed to match traditional Unix’s behavior (which is bizarre, but it’s too late to fix this). Please note that the same problem does arise when the empty string results from a computation, as in:
expr bar : foo \| foo : bar
Avoid this portability problem by avoiding the empty string.
expr
(‘:’)Portable expr
regular expressions should use ‘\’ to
escape only characters in the string ‘$()*.0123456789[\^n{}’.
For example, alternation, ‘\|’, is common but Posix does not
require its support, so it should be avoided in portable scripts.
Similarly, ‘\+’ and ‘\?’ should be avoided.
Portable expr
regular expressions should not begin with
‘^’. Patterns are automatically anchored so leading ‘^’ is
not needed anyway.
On the other hand, the behavior of the ‘$’ anchor is not portable on multi-line strings. Posix is ambiguous whether the anchor applies to each line, as was done in older versions of the GNU Core Utilities, or whether it applies only to the end of the overall string, as in Coreutils 6.0 and most other implementations.
$ baz='foo > bar' $ expr "X$baz" : 'X\(foo\)$' $ expr-5.97 "X$baz" : 'X\(foo\)$' foo
The Posix standard is ambiguous as to whether
‘expr 'a' : '\(b\)'’ outputs ‘0’ or the empty string.
In practice, it outputs the empty string on most platforms, but portable
scripts should not assume this. For instance, the QNX 4.25 native
expr
returns ‘0’.
One might think that a way to get a uniform behavior would be to use the empty string as a default value:
expr a : '\(b\)' \| ''
Unfortunately this behaves exactly as the original expression; see the
expr
(‘|’) entry for more information.
Some ancient expr
implementations (e.g., SunOS 4 expr
and
Solaris 8 /usr/ucb/expr
) have a silly length limit that causes
expr
to fail if the matched substring is longer than 120
bytes. In this case, you might want to fall back on ‘echo|sed’ if
expr
fails. Nowadays this is of practical importance only for
the rare installer who mistakenly puts /usr/ucb before
/usr/bin in PATH
.
On Mac OS X 10.4, expr
mishandles the pattern ‘[^-]’ in
some cases. For example, the command
expr Xpowerpc-apple-darwin8.1.0 : 'X[^-]*-[^-]*-\(.*\)'
outputs ‘apple-darwin8.1.0’ rather than the correct ‘darwin8.1.0’. This particular case can be worked around by substituting ‘[^--]’ for ‘[^-]’.
Don’t leave, there is some more!
The QNX 4.25 expr
, in addition of preferring ‘0’ to
the empty string, has a funny behavior in its exit status: it’s always 1
when parentheses are used!
$ val=`expr 'a' : 'a'`; echo "$?: $val" 0: 1 $ val=`expr 'a' : 'b'`; echo "$?: $val" 1: 0 $ val=`expr 'a' : '\(a\)'`; echo "?: $val" 1: a $ val=`expr 'a' : '\(b\)'`; echo "?: $val" 1: 0
In practice this can be a big problem if you are ready to catch failures
of expr
programs with some other method (such as using
sed
), since you may get twice the result. For instance
$ expr 'a' : '\(a\)' || echo 'a' | sed 's/^\(a\)$/\1/'
outputs ‘a’ on most hosts, but ‘aa’ on QNX 4.25. A
simple workaround consists of testing expr
and using a variable
set to expr
or to false
according to the result.
Tru64 expr
incorrectly treats the result as a number, if it
can be interpreted that way:
$ expr 00001 : '.*\(...\)' 1
On HP-UX 11, expr
only supports a single
sub-expression.
$ expr 'Xfoo' : 'X\(f\(oo\)*\)$' expr: More than one '\(' was used.
fgrep
Although Posix stopped requiring fgrep
in 2001,
a few traditional hosts (notably Solaris) do not support the Posix
replacement grep -F
. Also, some traditional implementations do
not work on long input lines. To work around these problems, invoke
AC_PROG_FGREP
and then use $FGREP
.
Tru64/OSF 5.1 fgrep
does not match an empty pattern.
find
The -maxdepth option seems to be GNU specific.
Tru64 v5.1, NetBSD 1.5 and Solaris find
commands do not understand it.
The replacement of ‘{}’ is guaranteed only if the argument is exactly {}, not if it’s only a part of an argument. For instance on DU, and HP-UX 10.20 and HP-UX 11:
$ touch foo $ find . -name foo -exec echo "{}-{}" \; {}-{}
while GNU find
reports ‘./foo-./foo’.
grep
Portable scripts can rely on the grep
options -c,
-l, -n, and -v, but should avoid other
options. For example, don’t use -w, as Posix does not require
it and Irix 6.5.16m’s grep
does not support it. Also,
portable scripts should not combine -c with -l,
as Posix does not allow this.
Some of the options required by Posix are not portable in practice.
Don’t use ‘grep -q’ to suppress output, because traditional grep
implementations (e.g., Solaris) do not support -q.
Don’t use ‘grep -s’ to suppress output either, because Posix
says -s does not suppress output, only some error messages;
also, the -s option of traditional grep
behaved
like -q does in most modern implementations. Instead,
redirect the standard output and standard error (in case the file
doesn’t exist) of grep
to /dev/null. Check the exit
status of grep
to determine whether it found a match.
The QNX4 implementation fails to count lines with grep -c '$'
,
but works with grep -c '^'
. Other alternatives for counting
lines are to use sed -n '$='
or wc -l
.
Some traditional grep
implementations do not work on long
input lines. On AIX the default grep
silently truncates long
lines on the input before matching.
Also, traditional implementations do not support multiple regexps
with -e: they either reject -e entirely (e.g., Solaris)
or honor only the last pattern (e.g., IRIX 6.5 and NeXT). To
work around these problems, invoke AC_PROG_GREP
and then use
$GREP
.
Another possible workaround for the multiple -e problem is to separate the patterns by newlines, for example:
grep 'foo bar' in.txt
except that this fails with traditional grep
implementations and with OpenBSD 3.8 grep
.
Traditional grep
implementations (e.g., Solaris) do not
support the -E or -F options. To work around these
problems, invoke AC_PROG_EGREP
and then use $EGREP
, and
similarly for AC_PROG_FGREP
and $FGREP
. Even if you are
willing to require support for Posix grep
, your script should
not use both -E and -F, since Posix does not allow
this combination.
Portable grep
regular expressions should use ‘\’ only to
escape characters in the string ‘$()*.0123456789[\^{}’. For example,
alternation, ‘\|’, is common but Posix does not require its
support in basic regular expressions, so it should be avoided in
portable scripts. Solaris and HP-UX grep
do not support it.
Similarly, the following escape sequences should also be avoided:
‘\<’, ‘\>’, ‘\+’, ‘\?’, ‘\`’, ‘\'’,
‘\B’, ‘\b’, ‘\S’, ‘\s’, ‘\W’, and ‘\w’.
Posix does not specify the behavior of grep
on binary files.
An example where this matters is using BSD grep
to
search text that includes embedded ANSI escape sequences for
colored output to terminals (‘\033[m’ is the sequence to restore
normal output); the behavior depends on whether input is seekable:
$ printf 'esc\033[mape\n' > sample $ grep . sample Binary file sample matches $ cat sample | grep . escape
join
Solaris 8 join
has bugs when the second operand is standard
input, and when standard input is a pipe. For example, the following
shell script causes Solaris 8 join
to loop forever:
cat >file <<'EOF' 1 x 2 y EOF cat file | join file -
Use ‘join - file’ instead.
On NetBSD, join -a 1 file1 file2
mistakenly behaves like
join -a 1 -a 2 1 file1 file2
, resulting in a usage warning;
the workaround is to use join -a1 file1 file2
instead.
ln
The -f option is portable nowadays.
Symbolic links are not available on some systems; use ‘$(LN_S)’ as a portable substitute.
For versions of the DJGPP before 2.04,
ln
emulates symbolic links
to executables by generating a stub that in turn calls the real
program. This feature also works with nonexistent files like in the
Posix spec. So ‘ln -s file link’ generates link.exe,
which attempts to call file.exe if run. But this feature only
works for executables, so ‘cp -p’ is used instead for these
systems. DJGPP versions 2.04 and later have full support
for symbolic links.
ls
The portable options are -acdilrtu. Current practice is for
-l to output both owner and group, even though ancient versions
of ls
omitted the group.
On ancient hosts, ‘ls foo’ sent the diagnostic ‘foo not found’
to standard output if foo did not exist. Hence a shell command
like ‘sources=`ls *.c 2>/dev/null`’ did not always work, since it
was equivalent to ‘sources='*.c not found'’ in the absence of
‘.c’ files. This is no longer a practical problem, since current
ls
implementations send diagnostics to standard error.
The behavior of ls
on a directory that is being concurrently
modified is not always predictable, because of a data race where cached
information returned by readdir
does not match the current
directory state. In fact, MacOS 10.5 has an intermittent bug where
readdir
, and thus ls
, sometimes lists a file more than
once if other files were added or removed from the directory immediately
prior to the ls
call. Since ls
already sorts its
output, the duplicate entries can be avoided by piping the results
through uniq
.
mkdir
No mkdir
option is portable to older systems. Instead of
‘mkdir -p file-name’, you should use
AS_MKDIR_P(file-name)
(see Programming in M4sh)
or AC_PROG_MKDIR_P
(see Particular Program Checks).
Combining the -m and -p options, as in ‘mkdir -m
go-w -p dir’, often leads to trouble. FreeBSD
mkdir
incorrectly attempts to change the permissions of
dir even if it already exists. HP-UX 11.23 and
IRIX 6.5 mkdir
often assign the wrong permissions to
any newly-created parents of dir.
Posix does not clearly specify whether ‘mkdir -p foo’
should succeed when foo is a symbolic link to an already-existing
directory. The GNU Core Utilities 5.1.0 mkdir
succeeds, but Solaris mkdir
fails.
Traditional mkdir -p
implementations suffer from race conditions.
For example, if you invoke mkdir -p a/b
and mkdir -p a/c
at the same time, both processes might detect that a is missing,
one might create a, then the other might try to create a
and fail with a File exists
diagnostic. The GNU Core
Utilities (‘fileutils’ version 4.1), FreeBSD 5.0,
NetBSD 2.0.2, and OpenBSD 2.4 are known to be
race-free when two processes invoke mkdir -p
simultaneously, but
earlier versions are vulnerable. Solaris mkdir
is still
vulnerable as of Solaris 10, and other traditional Unix systems are
probably vulnerable too. This possible race is harmful in parallel
builds when several Make rules call mkdir -p
to
construct directories. You may use
install-sh -d
as a safe replacement, provided this script is
recent enough; the copy shipped with Autoconf 2.60 and Automake 1.10 is
OK, but copies from older versions are vulnerable.
mkfifo
mknod
The GNU Coding Standards state that mknod
is safe to use on
platforms where it has been tested to exist; but it is generally portable
only for creating named FIFOs, since device numbers are
platform-specific. Autotest uses mkfifo
to implement parallel
testsuites. Posix states that behavior is unspecified when opening a
named FIFO for both reading and writing; on at least Cygwin, this
results in failure on any attempt to read or write to that file
descriptor.
mktemp
Shell scripts can use temporary files safely with mktemp
, but
it does not exist on all systems. A portable way to create a safe
temporary file name is to create a temporary directory with mode 700 and
use a file inside this directory. Both methods prevent attackers from
gaining control, though mktemp
is far less likely to fail
gratuitously under attack.
Here is sample code to create a new temporary directory ‘$dir’ safely:
# Create a temporary directory $dir in $TMPDIR (default /tmp). # Use mktemp if possible; otherwise fall back on mkdir, # with $RANDOM to make collisions less likely. : "${TMPDIR:=/tmp}" { dir=` (umask 077 && mktemp -d "$TMPDIR/fooXXXXXX") 2>/dev/null ` && test -d "$dir" } || { dir=$TMPDIR/foo$$-$RANDOM (umask 077 && mkdir "$dir") } || exit $?
mv
The only portable options are -f and -i.
Moving individual files between file systems is portable (it was in Unix version 6), but it is not always atomic: when doing ‘mv new existing’, there’s a critical section where neither the old nor the new version of existing actually exists.
On some systems moving files from /tmp can sometimes cause
undesirable (but perfectly valid) warnings, even if you created these
files. This is because /tmp belongs to a group that ordinary
users are not members of, and files created in /tmp inherit
the group of /tmp. When the file is copied, mv
issues
a diagnostic without failing:
$ touch /tmp/foo $ mv /tmp/foo . error→mv: ./foo: set owner/group (was: 100/0): Operation not permitted $ echo $? 0 $ ls foo foo
This annoying behavior conforms to Posix, unfortunately.
Moving directories across mount points is not portable, use cp
and rm
.
DOS variants cannot rename or remove open files, and do not support commands like ‘mv foo bar >foo’, even though this is perfectly portable among Posix hosts.
od
In MacOS X versions prior to 10.4.3, od
does not support the
standard Posix options -A, -j, -N, or
-t, or the XSI option, -s. The only
supported Posix option is -v, and the only supported
XSI options are those in -bcdox. The BSD
hexdump
program can be used instead.
In some versions of some operating systems derived from Solaris 11,
od
prints decimal byte values padded with zeroes rather than
with spaces:
$ printf '#!' | od -A n -t d1 -N 2 035 033
instead of
$ printf '#!' | od -A n -t d1 -N 2 35 33
We have observed this on both OpenIndiana and OmniOS;
Illumos may also be affected.
As a workaround, you can use octal output (option -t o1
).
rm
The -f and -r options are portable.
It is not portable to invoke rm
without options or operands.
On the other hand, Posix now requires rm -f
to silently
succeed when there are no operands (useful for constructs like
rm -rf $filelist
without first checking if ‘$filelist’
was empty). But this was not always portable; at least NetBSD
rm
built before 2008 would fail with a diagnostic.
A file might not be removed even if its parent directory is writable and searchable. Many Posix hosts cannot remove a mount point, a named stream, a working directory, or a last link to a file that is being executed.
DOS variants cannot rename or remove open files, and do not support commands like ‘rm foo >foo’, even though this is perfectly portable among Posix hosts.
rmdir
Just as with rm
, some platforms refuse to remove a working
directory.
sed
Patterns should not include the separator (unless escaped), even as part
of a character class. In conformance with Posix, the Cray
sed
rejects ‘s/[^/]*$//’: use ‘s%[^/]*$%%’.
Even when escaped, patterns should not include separators that are also
used as sed
metacharacters. For example, GNU sed 4.0.9 rejects
‘s,x\{1\,\},,’, while sed 4.1 strips the backslash before the comma
before evaluating the basic regular expression.
Avoid empty patterns within parentheses (i.e., ‘\(\)’). Posix does
not require support for empty patterns, and Unicos 9 sed
rejects
them.
Unicos 9 sed
loops endlessly on patterns like ‘.*\n.*’.
Sed scripts should not use branch labels longer than 7 characters and
should not contain comments; AIX 5.3 sed
rejects indented comments.
HP-UX sed has a limit of 99 commands (not counting ‘:’ commands) and
48 labels, which cannot be circumvented by using more than one script
file. It can execute up to 19 reads with the ‘r’ command per cycle.
Solaris /usr/ucb/sed
rejects usages that exceed a limit of
about 6000 bytes for the internal representation of commands.
Avoid redundant ‘;’, as some sed
implementations, such as
NetBSD 1.4.2’s, incorrectly try to interpret the second
‘;’ as a command:
$ echo a | sed 's/x/x/;;s/x/x/' sed: 1: "s/x/x/;;s/x/x/": invalid command code ;
Some sed
implementations have a buffer limited to 4000 bytes,
and this limits the size of input lines, output lines, and internal
buffers that can be processed portably. Likewise,
not all sed
implementations can handle embedded NUL
or
a missing trailing newline.
Remember that ranges within a bracket expression of a regular expression
are only well-defined in the ‘C’ (or ‘POSIX’) locale.
Meanwhile, support for character classes like ‘[[:upper:]]’ is not
yet universal, so if you cannot guarantee the setting of LC_ALL
,
it is better to spell out a range ‘[ABCDEFGHIJKLMNOPQRSTUVWXYZ]’
than to rely on ‘[A-Z]’.
Additionally, Posix states that regular expressions are only well-defined on characters. Unfortunately, there exist platforms such as MacOS X 10.5 where not all 8-bit byte values are valid characters, even though that platform has a single-byte ‘C’ locale. And Posix allows the existence of a multi-byte ‘C’ locale, although that does not yet appear to be a common implementation. At any rate, it means that not all bytes will be matched by the regular expression ‘.’:
$ printf '\200\n' | LC_ALL=C sed -n /./p | wc -l 0 $ printf '\200\n' | LC_ALL=en_US.ISO8859-1 sed -n /./p | wc -l 1
Portable sed
regular expressions should use ‘\’ only to escape
characters in the string ‘$()*.0123456789[\^n{}’. For example,
alternation, ‘\|’, is common but Posix does not require its
support, so it should be avoided in portable scripts. Solaris
sed
does not support alternation; e.g., ‘sed '/a\|b/d'’
deletes only lines that contain the literal string ‘a|b’.
Similarly, ‘\+’ and ‘\?’ should be avoided.
Anchors (‘^’ and ‘$’) inside groups are not portable.
Nested parentheses in patterns (e.g., ‘\(\(a*\)b*)\)’) are
quite portable to current hosts, but was not supported by some ancient
sed
implementations like SVR3.
Some sed
implementations, e.g., Solaris, restrict the special
role of the asterisk ‘*’ to one-character regular expressions and
back-references, and the special role of interval expressions
‘\{m\}’, ‘\{m,\}’, or ‘\{m,n\}’
to one-character regular expressions. This may lead to unexpected behavior:
$ echo '1*23*4' | /usr/bin/sed 's/\(.\)*/x/g' x2x4 $ echo '1*23*4' | /usr/xpg4/bin/sed 's/\(.\)*/x/g' x
The -e option is mostly portable. However, its argument cannot start with ‘a’, ‘c’, or ‘i’, as this runs afoul of a Tru64 5.1 bug. Also, its argument cannot be empty, as this fails on AIX 5.3. Some people prefer to use ‘-e’:
sed -e 'command-1' \ -e 'command-2'
as opposed to the equivalent:
sed ' command-1 command-2 '
The following usage is sometimes equivalent:
sed 'command-1;command-2'
but Posix says that this use of a semicolon has undefined effect if command-1’s verb is ‘{’, ‘a’, ‘b’, ‘c’, ‘i’, ‘r’, ‘t’, ‘w’, ‘:’, or ‘#’, so you should use semicolon only with simple scripts that do not use these verbs.
Posix up to the 2008 revision requires the argument of the -e
option to be a syntactically complete script. GNU sed
allows
to pass multiple script fragments, each as argument of a separate
-e option, that are then combined, with newlines between the
fragments, and a future Posix revision may allow this as well. This
approach is not portable with script fragments ending in backslash; for
example, the sed
programs on Solaris 10, HP-UX 11, and AIX
don’t allow splitting in this case:
$ echo a | sed -n -e 'i\ 0' 0 $ echo a | sed -n -e 'i\' -e 0 Unrecognized command: 0
In practice, however, this technique of joining fragments
through -e works for multiple sed
functions within
‘{’ and ‘}’, even if that is not specified by Posix:
$ echo a | sed -n -e '/a/{' -e s/a/b/ -e p -e '}' b
Commands inside { } brackets are further restricted. Posix 2008 says that they cannot be preceded by addresses, ‘!’, or ‘;’, and that each command must be followed immediately by a newline, without any intervening blanks or semicolons. The closing bracket must be alone on a line, other than white space preceding or following it. However, a future version of Posix may standardize the use of addresses within brackets.
Contrary to yet another urban legend, you may portably use ‘&’ in
the replacement part of the s
command to mean “what was
matched”. All descendants of Unix version 7 sed
(at least; we
don’t have first hand experience with older sed
implementations) have
supported it.
Posix requires that you must not have any white space between ‘!’ and the following command. It is OK to have blanks between the address and the ‘!’. For instance, on Solaris:
$ echo "foo" | sed -n '/bar/ ! p' error→Unrecognized command: /bar/ ! p $ echo "foo" | sed -n '/bar/! p' error→Unrecognized command: /bar/! p $ echo "foo" | sed -n '/bar/ !p' foo
Posix also says that you should not combine ‘!’ and ‘;’. If you use ‘!’, it is best to put it on a command that is delimited by newlines rather than ‘;’.
Also note that Posix requires that the ‘b’, ‘t’, ‘r’, and ‘w’ commands be followed by exactly one space before their argument. On the other hand, no white space is allowed between ‘:’ and the subsequent label name.
If a sed script is specified on the command line and ends in an
‘a’, ‘c’, or ‘i’ command, the last line of inserted text
should be followed by a newline. Otherwise some sed
implementations (e.g., OpenBSD 3.9) do not append a newline to the
inserted text.
Many sed
implementations (e.g., MacOS X 10.4,
OpenBSD 3.9, Solaris 10
/usr/ucb/sed
) strip leading white space from the text of
‘a’, ‘c’, and ‘i’ commands. Prepend a backslash to
work around this incompatibility with Posix:
$ echo flushleft | sed 'a\ > indented > ' flushleft indented $ echo foo | sed 'a\ > \ indented > ' flushleft indented
Posix requires that with an empty regular expression, the last non-empty regular expression from either an address specification or substitution command is applied. However, busybox 1.6.1 complains when using a substitution command with a replacement containing a back-reference to an empty regular expression; the workaround is repeating the regular expression.
$ echo abc | busybox sed '/a\(b\)c/ s//\1/' sed: No previous regexp. $ echo abc | busybox sed '/a\(b\)c/ s/a\(b\)c/\1/' b
Portable scripts should be aware of the inconsistencies and options for handling word boundaries, as these are not specified by POSIX.
\< \b [[:<:]] Solaris 10 yes no no Solaris XPG4 yes no error NetBSD 5.1 no no yes FreeBSD 9.1 no no yes GNU yes yes error busybox yes yes error
sed
(‘t’)Some old systems have sed
that “forget” to reset their
‘t’ flag when starting a new cycle.  When processing line 1, (c) matches, therefore sets the ‘t’
flag, and the output is produced. When processing
line 2, the ‘t’ flag is still set (this is the bug). Command (a)
fails to match, but sed
is not supposed to clear the ‘t’
flag when a substitution fails. Command (b) sees that the flag is set,
therefore it clears it, and jumps to (d), hence you get ‘delete me’
instead of ‘deleted’. When processing line (3), ‘t’ is clear,
(a) matches, so the flag is set, hence (b) clears the flags and jumps.
Finally, since the flag is clear, line 4 is processed properly.
There are two things one should remember about ‘t’ in sed
.
Firstly, always remember that ‘t’ jumps if some substitution
succeeded, not only the immediately preceding substitution. Therefore,
always use a fake ‘t clear’ followed by a ‘:clear’ on the next
line, to reset the ‘t’ flag where needed.
Secondly, you cannot rely on sed
to clear the flag at each new
cycle.
One portable implementation of the script above is:
t clear :clear s/keep me/kept/g t end s/.*/deleted/g :end
sleep
Using sleep
is generally portable. However, remember that
adding a sleep
to work around timestamp issues, with a minimum
granularity of one second, doesn’t scale well for parallel builds on
modern machines with sub-second process completion.
sort
Remember that sort order is influenced by the current locale. Inside
configure, the C locale is in effect, but in Makefile snippets,
you may need to specify LC_ALL=C sort
.
tar
There are multiple file formats for tar
; if you use Automake,
the macro AM_INIT_AUTOMAKE
has some options controlling which
level of portability to use.
touch
If you specify the desired timestamp (e.g., with the -r
option), older touch
implementations use the utime
or
utimes
system call, which can result in the same kind of
timestamp truncation problems that ‘cp -p’ has.
On ancient BSD systems, touch
or any command that
results in an empty file does not update the timestamps, so use a
command like echo
as a workaround.
Also,
GNU touch
3.16r (and presumably all before that)
fails to work on SunOS 4.1.3 when the empty file is on an
NFS-mounted 4.2 volume.
However, these problems are no longer of practical concern.
tr
Not all versions of tr
handle all backslash character escapes.
For example, Solaris 10 /usr/ucb/tr
falls over, even though
Solaris contains more modern tr
in other locations.
Using octal escapes is more portable for carriage returns, since
‘\015’ is the same for both ASCII and EBCDIC, and since use of
literal carriage returns in scripts causes a number of other problems.
But for other characters, like newline, using octal escapes ties the
operation to ASCII, so it is better to use literal characters.
$ { echo moon; echo light; } | /usr/ucb/tr -d '\n' ; echo moo light $ { echo moon; echo light; } | /usr/bin/tr -d '\n' ; echo moonlight $ { echo moon; echo light; } | /usr/ucb/tr -d '\012' ; echo moonlight $ nl=' '; { echo moon; echo light; } | /usr/ucb/tr -d "$nl" ; echo moonlight
Not all versions of tr
recognize direct ranges of characters: at
least Solaris /usr/bin/tr
still fails to do so. But you can
use /usr/xpg4/bin/tr
instead, or add brackets (which in Posix
transliterate to themselves).
$ echo "Hazy Fantazy" | LC_ALL=C /usr/bin/tr a-z A-Z HAZy FAntAZy $ echo "Hazy Fantazy" | LC_ALL=C /usr/bin/tr '[a-z]' '[A-Z]' HAZY FANTAZY $ echo "Hazy Fantazy" | LC_ALL=C /usr/xpg4/bin/tr a-z A-Z HAZY FANTAZY
When providing two arguments, be sure the second string is at least as long as the first.
$ echo abc | /usr/xpg4/bin/tr bc d adc $ echo abc | coreutils/tr bc d add
Posix requires tr
to operate on binary files. But at least
Solaris /usr/ucb/tr
and /usr/bin/tr
silently discard
NUL
in the input prior to doing any translation. When using
tr
to process a binary file that may contain NUL
bytes,
it is necessary to use /usr/xpg4/bin/tr
instead, or
/usr/xpg6/bin/tr
if that is available.
$ printf 'a\0b' | /usr/ucb/tr x x | od -An -tx1 61 62 $ printf 'a\0b' | /usr/bin/tr x x | od -An -tx1 61 62 $ printf 'a\0b' | /usr/xpg4/bin/tr x x | od -An -tx1 61 00 62
Solaris /usr/ucb/tr
additionally fails to handle ‘\0’ as the
octal escape for NUL
.
$ printf 'abc' | /usr/ucb/tr 'bc' '\0d' | od -An -tx1 61 62 63 $ printf 'abc' | /usr/bin/tr 'bc' '\0d' | od -An -tx1 61 00 64 $ printf 'abc' | /usr/xpg4/bin/tr 'bc' '\0d' | od -An -tx1 61 00 64
Next: Portable C and C++ Programming, Previous: Portable Shell Programming, Up: Autoconf [Contents][Index]
Writing portable makefiles is an art. Since a makefile’s commands are
executed by the shell, you must consider the shell portability issues
already mentioned. However, other issues are specific to make
itself.
$<
in Ordinary Make Rulesmake macro=value
and SubmakesSHELL
make -k
VPATH
and MakeNext: Failure in Make Rules, Up: Portable Make Programming [Contents][Index]
$<
in Ordinary Make RulesPosix says that the ‘$<’ construct in makefiles can be
used only in inference rules and in the ‘.DEFAULT’ rule; its
meaning in ordinary rules is unspecified. Solaris make
for instance replaces it with the empty string. OpenBSD (3.0 and
later) make
diagnoses these uses and errors out.
Next: Special Characters in Make Macro Names, Previous: $<
in Ordinary Make Rules, Up: Portable Make Programming [Contents][Index]
Posix 2008 requires that make
must invoke each command with
the equivalent of a ‘sh -e -c’ subshell, which causes the
subshell to exit immediately if a subsidiary simple-command fails,
although not all make
implementations have historically
followed this rule. For
example, the command ‘touch T; rm -f U’ may attempt to
remove U even if the touch
fails, although this is not
permitted with Posix make. One way to work around failures in simple
commands is to reword them so that they always succeed, e.g., ‘touch
T || :; rm -f U’.
However, even this approach can run into common bugs in BSD
implementations of the -e option of sh
and
set
(see Limitations of Shell Builtins), so if you
are worried
about porting to buggy BSD shells it may be simpler to migrate
complicated make
actions into separate scripts.
Next: Backslash-Newline Before Empty Lines, Previous: Failure in Make Rules, Up: Portable Make Programming [Contents][Index]
Posix limits macro names to nonempty strings containing only
ASCII letters and digits, ‘.’, and ‘_’. Many
make
implementations allow a wider variety of characters, but
portable makefiles should avoid them. It is portable to start a name
with a special character, e.g., ‘$(.FOO)’.
Some ancient make
implementations don’t support leading
underscores in macro names. An example is NEWS-OS 4.2R.
$ cat Makefile _am_include = # _am_quote = all:; @echo this is test $ make Make: Must be a separator on rules line 2. Stop. $ cat Makefile2 am_include = # am_quote = all:; @echo this is test $ make -f Makefile2 this is test
However, this problem is no longer of practical concern.
Next: Backslash-Newline in Make Comments, Previous: Special Characters in Make Macro Names, Up: Portable Make Programming [Contents][Index]
A bug in Bash 2.03 can cause problems if a Make rule contains a backslash-newline followed by line that expands to nothing. For example, on Solaris 8:
SHELL = /bin/bash EMPTY = foo: touch foo \ $(EMPTY)
executes
/bin/bash -c 'touch foo \ '
which fails with a syntax error, due to the Bash bug. To avoid this problem, avoid nullable macros in the last line of a multi-line command.
On some versions of HP-UX, make
reads multiple newlines
following a backslash, continuing to the next non-empty line. For
example,
FOO = one \ BAR = two test: : FOO is "$(FOO)" : BAR is "$(BAR)"
shows FOO
equal to one BAR = two
. Other implementations
sensibly let a backslash continue only to the immediately following
line.
Next: Long Lines in Makefiles, Previous: Backslash-Newline Before Empty Lines, Up: Portable Make Programming [Contents][Index]
According to Posix, Make comments start with #
and continue until an unescaped newline is reached.
$ cat Makefile # A = foo \ bar \ baz all: @echo ok $ make # GNU make ok
However this is not always the case. Some implementations
discard everything from #
through the end of the line, ignoring any
trailing backslash.
$ pmake # BSD make "Makefile", line 3: Need an operator Fatal errors encountered -- cannot continue
Therefore, if you want to comment out a multi-line definition, prefix each
line with #
, not only the first.
# A = foo \ # bar \ # baz
Next: make macro=value
and Submakes, Previous: Backslash-Newline in Make Comments, Up: Portable Make Programming [Contents][Index]
Tru64 5.1’s make
has been reported to crash when given a
makefile with lines longer than around 20 kB. Earlier versions are
reported to exit with Line too long
diagnostics.
Next: The Make Macro MAKEFLAGS, Previous: Long Lines in Makefiles, Up: Portable Make Programming [Contents][Index]
make macro=value
and SubmakesA command-line variable definition such as foo=bar
overrides any
definition of foo
in a makefile. Some make
implementations (such as GNU make
) propagate this
override to subsidiary invocations of make
. Some other
implementations do not pass the substitution along to submakes.
$ cat Makefile foo = foo one: @echo $(foo) $(MAKE) two two: @echo $(foo) $ make foo=bar # GNU make 3.79.1 bar make two make[1]: Entering directory `/home/adl' bar make[1]: Leaving directory `/home/adl' $ pmake foo=bar # BSD make bar pmake two foo
You have a few possibilities if you do want the foo=bar
override
to propagate to submakes. One is to use the -e
option, which causes all environment variables to have precedence over
the makefile macro definitions, and declare foo as an environment
variable:
$ env foo=bar make -e
The -e option is propagated to submakes automatically,
and since the environment is inherited between make
invocations, the foo
macro is overridden in
submakes as expected.
This syntax (foo=bar make -e
) is portable only when used
outside of a makefile, for instance from a script or from the
command line. When run inside a make
rule, GNU
make
3.80 and prior versions forget to propagate the
-e option to submakes.
Moreover, using -e could have unexpected side effects if your
environment contains some other macros usually defined by the
makefile. (See also the note about make -e
and SHELL
below.)
If you can foresee all macros that a user might want to override, then you can propagate them to submakes manually, from your makefile:
foo = foo one: @echo $(foo) $(MAKE) foo=$(foo) two two: @echo $(foo)
Another way to propagate a variable to submakes in a portable way is to expand an extra variable in every invocation of ‘$(MAKE)’ within your makefile:
foo = foo one: @echo $(foo) $(MAKE) $(SUBMAKEFLAGS) two two: @echo $(foo)
Users must be aware that this technique is in use to take advantage of
it, e.g. with make foo=bar SUBMAKEFLAGS='foo=bar'
, but it
allows any macro to be overridden. Makefiles generated by
automake
use this technique, expanding $(AM_MAKEFLAGS)
on the command lines of submakes (see Automake in GNU Automake).
Next: The Make Macro SHELL
, Previous: make macro=value
and Submakes, Up: Portable Make Programming [Contents][Index]
Posix requires make
to use MAKEFLAGS
to affect the
current and recursive invocations of make, but allows implementations
several formats for the variable. It is tricky to parse
$MAKEFLAGS
to determine whether -s for silent execution
or -k for continued execution are in effect. For example, you
cannot assume that the first space-separated word in $MAKEFLAGS
contains single-letter options, since in the Cygwin version of
GNU make
it is either --unix or
--win32 with the second word containing single-letter options.
$ cat Makefile all: @echo MAKEFLAGS = $(MAKEFLAGS) $ make MAKEFLAGS = --unix $ make -k MAKEFLAGS = --unix -k
Next: Parallel Make, Previous: The Make Macro MAKEFLAGS, Up: Portable Make Programming [Contents][Index]
SHELL
Posix-compliant make
internally uses the $(SHELL)
macro to spawn shell processes and execute Make rules. This
is a builtin macro supplied by make
, but it can be modified
by a makefile or by a command-line argument.
Not all make
implementations define this SHELL
macro.
Tru64
make
is an example; this implementation always uses
/bin/sh
. So it’s a good idea to always define SHELL
in
your makefiles. If you use Autoconf, do
SHELL = @SHELL@
If you use Automake, this is done for you.
Do not force SHELL = /bin/sh
because that is not correct
everywhere. Remember, /bin/sh is not Posix compliant on many
systems, such as FreeBSD 4, NetBSD 3, AIX 3, Solaris 10, or Tru64.
Additionally, DJGPP lacks /bin/sh
, and when its
GNU make
port sees such a setting it enters a
special emulation mode where features like pipes and redirections are
emulated on top of DOS’s command.com
. Unfortunately this
emulation is incomplete; for instance it does not handle command
substitutions. Using @SHELL@
means that your makefile will
benefit from the same improved shell, such as bash
or
ksh
, that was discovered during configure
, so that
you aren’t fighting two different sets of shell bugs between the two
contexts.
Posix-compliant make
should never acquire the value of
$(SHELL) from the environment, even when make -e
is used
(otherwise, think about what would happen to your rules if
SHELL=/bin/tcsh
).
However not all make
implementations have this exception.
For instance it’s not surprising that Tru64 make
doesn’t
protect SHELL
, since it doesn’t use it.
$ cat Makefile SHELL = /bin/sh FOO = foo all: @echo $(SHELL) @echo $(FOO) $ env SHELL=/bin/tcsh FOO=bar make -e # Tru64 Make /bin/tcsh bar $ env SHELL=/bin/tcsh FOO=bar gmake -e # GNU make /bin/sh bar
Conversely, make
is not supposed to export any changes to the
macro SHELL
to child processes. Again, many implementations
break this rule:
$ cat Makefile all: @echo $(SHELL) @printenv SHELL $ env SHELL=sh make -e SHELL=/bin/ksh # BSD Make, GNU make 3.80 /bin/ksh /bin/ksh $ env SHELL=sh gmake -e SHELL=/bin/ksh # GNU make 3.81 /bin/ksh sh
Next: Comments in Make Rules, Previous: The Make Macro SHELL
, Up: Portable Make Programming [Contents][Index]
Support for parallel execution in make
implementation varies.
Generally, using GNU make is your best bet.
When NetBSD or FreeBSD make
are run in parallel mode, they will
reuse the same shell for multiple commands within one recipe. This can
have various unexpected consequences. For example, changes of directories
or variables persist between recipes, so that:
all: @var=value; cd /; pwd; echo $$var; echo $$$$ @pwd; echo $$var; echo $$$$
may output the following with make -j1
, at least on NetBSD up to
5.1 and FreeBSD up to 8.2:
/ value 32235 / value 32235
while without -j1, or with -B, the output looks less surprising:
/ value 32238 /tmp 32239
Another consequence is that, if one command in a recipe uses exit 0
to indicate a successful exit, the shell will be gone and the remaining
commands of this recipe will not be executed.
The BSD make
implementations, when run in parallel mode,
will also pass the Makefile
recipes to the shell through
its standard input, thus making it unusable from the recipes:
$ cat Makefile read: @read line; echo LINE: $$line $ echo foo | make read LINE: foo $ echo foo | make -j1 read # NetBSD 5.1 and FreeBSD 8.2 LINE:
Moreover, when FreeBSD make
(up at least to 8.2) is run in
parallel mode, it implements the @
and -
“recipe
modifiers” by dynamically modifying the active shell flags. This
behavior has the effects of potentially clobbering the exit status
of recipes silenced with the @
modifier if they also unset
the errexit shell flag, and of mangling the output in
unexpected ways:
$ cat Makefile a: @echo $$-; set +e; false b: -echo $$-; false; echo set - $ make a; echo status: $? ehBc *** Error code 1 status: 1 $ make -j1 a; echo status: $? ehB status: 0 $ make b echo $-; echo set - hBc set - $ make -j1 b echo $-; echo hvB
You can avoid all these issues by using the -B option to enable compatibility semantics. However, that will effectively also disable all parallelism as that will cause prerequisites to be updated in the order they are listed in a rule.
Some make implementations (among them, FreeBSD make
, NetBSD
make
, and Solaris dmake
), when invoked with a
-jN option, connect the standard output and standard
error of all their child processes to pipes or temporary regular
files. This can lead to subtly different semantics in the behavior
of the spawned processes. For example, even if the make
standard output is connected to a tty, the recipe command will not be:
$ cat Makefile all: @test -t 1 && echo "Is a tty" || echo "Is not a tty" $ make -j 2 # FreeBSD 8.2 make Is not a tty $ make -j 2 # NetBSD 5.1 make --- all --- Is not a tty $ dmake -j 2 # Solaris 10 dmake hostname --> 1 job hostname --> Job output Is not a tty
On the other hand:
$ make -j 2 # GNU make, Heirloom make Is a tty
The above examples also show additional status output produced in parallel
mode for targets being updated by Solaris dmake
and NetBSD
make
(but not by FreeBSD make
).
Furthermore, parallel runs of those make
implementations will
route standard error from commands that they spawn into their own
standard output, and may remove leading whitespace from output lines.
Next: Newlines in Make Rules, Previous: Parallel Make, Up: Portable Make Programming [Contents][Index]
Never put comments in a rule.
Some make
treat anything starting with a tab as a command for
the current rule, even if the tab is immediately followed by a #
.
The make
from Tru64 Unix V5.1 is one of them. The following
makefile runs # foo
through the shell.
all: # foo
As a workaround, you can use the :
no-op command with a string
argument that gets ignored:
all: : "foo"
Conversely, if you want to use the ‘#’ character in some command,
you can only do so by expanding it inside a rule (see Comments in Make Macros). So for example, if ‘COMMENT_CHAR’ is substituted by
config.status
as ‘#’, then the following substitutes
‘@COMMENT_CHAR@’ in a generated header:
foo.h: foo.h.in sed -e 's|@''COMMENT_CHAR''@|@COMMENT_CHAR@|g' \ $(srcdir)/foo.h.in > $@
The funny shell quoting avoids a substitution at config.status
run time of the left-hand side of the sed
‘s’ command.
Next: Comments in Make Macros, Previous: Comments in Make Rules, Up: Portable Make Programming [Contents][Index]
In shell scripts, newlines can be used inside string literals. But in
the shell statements of Makefile rules, this is not possible:
A newline not preceded by a backslash is a separator between shell
statements. Whereas a newline that is preceded by a backslash becomes
part of the shell statement according to POSIX, but gets replaced,
together with the backslash that precedes it, by a space in GNU
make
3.80 and older. So, how can a newline be used in a string
literal?
The trick is to set up a shell variable that contains a newline:
nlinit=`echo 'nl="'; echo '"'`; eval "$$nlinit"
For example, in order to create a multi-line ‘sed’ expression that inserts a blank line after every line of a file, this code can be used:
nlinit=`echo 'nl="'; echo '"'`; eval "$$nlinit"; \ sed -e "s/\$$/\\$${nl}/" < input > output
Next: Trailing whitespace in Make Macros, Previous: Newlines in Make Rules, Up: Portable Make Programming [Contents][Index]
Avoid putting comments in macro values as far as possible. Posix specifies that the text starting from the ‘#’ sign until the end of the line is to be ignored, which has the unfortunate effect of disallowing them even within quotes. Thus, the following might lead to a syntax error at compile time:
CPPFLAGS = "-DCOMMENT_CHAR='#'"
as ‘CPPFLAGS’ may be expanded to ‘"-DCOMMENT_CHAR='’.
Most make
implementations disregard this and treat single and
double quotes specially here. Also, GNU make
lets you put
‘#’ into a macro value by escaping it with a backslash, i.e.,
‘\#’. However, neither of these usages are portable.
See Comments in Make Rules, for a portable alternative.
Even without quoting involved, comments can have surprising effects, because the whitespace before them is part of the variable value:
foo = bar # trailing comment print: ; @echo "$(foo)."
prints ‘bar .’, which is usually not intended, and can expose
make
bugs as described below.
Next: Command-line Macros and whitespace, Previous: Comments in Make Macros, Up: Portable Make Programming [Contents][Index]
GNU make
3.80 mistreats trailing whitespace in macro
substitutions and appends another spurious suffix:
empty = foo = bar $(empty) print: ; @echo $(foo:=.test)
prints ‘bar.test .test’.
BSD and Solaris make
implementations do not honor trailing
whitespace in macro definitions as Posix requires:
foo = bar # Note the space after "bar". print: ; @echo $(foo)t
prints ‘bart’ instead of ‘bar t’. To work around this, you can use a helper macro as in the previous example.
Next: The obj/ Subdirectory and Make, Previous: Trailing whitespace in Make Macros, Up: Portable Make Programming [Contents][Index]
Some make
implementations may strip trailing whitespace off
of macros set on the command line in addition to leading whitespace.

If an obj/ directory exists, BSD make
enters it
before reading the makefile. Hence the makefile in the
current directory is not read.
$ cat Makefile all: echo Hello $ cat obj/Makefile all: echo World $ make # GNU make echo Hello Hello $ pmake # BSD make echo World World
Next: VPATH
and Make, Previous: The obj/ Subdirectory and Make, Up: Portable Make Programming [Contents][Index]
make -k
Do not rely on the exit status of make -k
. Some implementations
reflect whether they encountered an error in their exit status; other
implementations always succeed.
$ cat Makefile all: false $ make -k; echo exit status: $? # GNU make false make: *** [all] Error 1 exit status: 2 $ pmake -k; echo exit status: $? # BSD make false *** Error code 1 (continuing) exit status: 0
Next: Single Suffix Rules and Separated Dependencies, Previous: Exit Status of make -k
, Up: Portable Make Programming [Contents][Index]
VPATH
and MakePosix does not specify the semantics of VPATH
. Typically,
make
supports VPATH
, but its implementation is not
consistent.
Autoconf and Automake support makefiles whose usages of VPATH
are
portable to recent-enough popular implementations of make
, but
to keep the resulting makefiles portable, a package’s makefile
prototypes must take the following issues into account. These issues
are complicated and are often poorly understood, and installers who use
VPATH
should expect to find many bugs in this area. If you use
VPATH
, the simplest way to avoid these portability bugs is to
stick with GNU make
, since it is the most
commonly-used make
among Autoconf users.
Here are some known issues with some VPATH
implementations.
VPATH
VPATH
and Double-colon Rules$<
Not Supported in Explicit Rulesmake
Creates Prerequisite Directories Magically
Next: VPATH
and Double-colon Rules, Up: VPATH
and Make [Contents][Index]
VPATH
Do not set VPATH
to the value of another variable, for example
‘VPATH = $(srcdir)’, because some ancient versions of
make
do not do variable substitutions on the value of
VPATH
. For example, use this
srcdir = @srcdir@ VPATH = @srcdir@
rather than ‘VPATH = $(srcdir)’. Note that with GNU Automake, there is no need to set this yourself.
Next: $<
Not Supported in Explicit Rules, Previous: Variables listed in VPATH
, Up: VPATH
and Make [Contents][Index]
VPATH
and Double-colon RulesWith ancient versions of Sun make
,
any assignment to VPATH
causes make
to execute only
the first set of double-colon rules.
However, this problem is no longer of practical concern.
Next: Automatic Rule Rewriting, Previous: VPATH
and Double-colon Rules, Up: VPATH
and Make [Contents][Index]
$<
Not Supported in Explicit RulesUsing $<
in explicit rules is not portable.
The prerequisite file must be named explicitly in the rule. If you want
to find the prerequisite via a VPATH
search, you have to code the
whole thing manually. See Build Directories.
Next: Tru64 make
Creates Prerequisite Directories Magically, Previous: $<
Not Supported in Explicit Rules, Up: VPATH
and Make [Contents][Index]
Some make
implementations, such as Solaris and Tru64,
search for prerequisites in VPATH
and
then rewrite each occurrence as a plain word in the rule.
For instance:
# This isn't portable to GNU make. VPATH = ../pkg/src f.c: if.c cp if.c f.c
executes cp ../pkg/src/if.c f.c
if if.c is
found in ../pkg/src.
However, this rule leads to real problems in practice. For example, if
the source directory contains an ordinary file named test that is
used in a dependency, Solaris make
rewrites commands like
‘if test -r foo; …’ to ‘if ../pkg/src/test -r foo;
…’, which is typically undesirable. In fact, make
is
completely unaware of shell syntax used in the rules, so the VPATH
rewrite can potentially apply to any whitespace-separated word
in a rule, including shell variables, functions, and keywords.
$ mkdir build $ cd build $ cat > Makefile <<'END' VPATH = .. all: arg func for echo func () { for arg in "$$@"; do echo $$arg; done; }; \ func "hello world" END $ touch ../arg ../func ../for ../echo $ make ../func () { ../for ../arg in "$@"; do ../echo $arg; done; }; \ ../func "hello world" sh: syntax error at line 1: `do' unexpected *** Error code 2
To avoid this problem, portable makefiles should never mention a source
file or dependency whose name is that of a shell keyword like for
or until, a shell command like cat
or gcc
or
test
, or a shell function or variable used in the corresponding
Makefile
recipe.
Because of these problems GNU make
and many other make
implementations do not rewrite commands, so portable makefiles should
search VPATH
manually. It is tempting to write this:
# This isn't portable to Solaris make. VPATH = ../pkg/src f.c: if.c cp `test -f if.c || echo $(VPATH)/`if.c f.c
However, the “prerequisite rewriting” still applies here. So if
if.c is in ../pkg/src, Solaris and Tru64 make
execute
cp `test -f ../pkg/src/if.c || echo ../pkg/src/`if.c f.c
which reduces to
cp if.c f.c
and thus fails. Oops.
A simple workaround, and good practice anyway, is to use ‘$?’ and ‘$@’ when possible:
VPATH = ../pkg/src f.c: if.c cp $? $@
but this does not generalize well to commands with multiple prerequisites. A more general workaround is to rewrite the rule so that the prerequisite if.c never appears as a plain word. For example, these three rules would be safe, assuming if.c is in ../pkg/src and the other files are in the working directory:
VPATH = ../pkg/src f.c: if.c f1.c cat `test -f ./if.c || echo $(VPATH)/`if.c f1.c >$@ g.c: if.c g1.c cat `test -f 'if.c' || echo $(VPATH)/`if.c g1.c >$@ h.c: if.c h1.c cat `test -f "if.c" || echo $(VPATH)/`if.c h1.c >$@
Things get worse when your prerequisites are in a macro.
VPATH = ../pkg/src HEADERS = f.h g.h h.h install-HEADERS: $(HEADERS) for i in $(HEADERS); do \ $(INSTALL) -m 644 \ `test -f $$i || echo $(VPATH)/`$$i \ $(DESTDIR)$(includedir)/$$i; \ done
The above install-HEADERS
rule is not Solaris-proof because for
i in $(HEADERS);
is expanded to for i in f.h g.h h.h;
where f.h
and g.h
are plain words and are hence
subject to VPATH
adjustments.
If the three files are in ../pkg/src, the rule is run as:
for i in ../pkg/src/f.h ../pkg/src/g.h h.h; do \ install -m 644 \ `test -f $i || echo ../pkg/src/`$i \ /usr/local/include/$i; \ done
where the two first install
calls fail. For instance,
consider the f.h
installation:
install -m 644 \ `test -f ../pkg/src/f.h || \ echo ../pkg/src/ \ `../pkg/src/f.h \ /usr/local/include/../pkg/src/f.h;
It reduces to:
install -m 644 \ ../pkg/src/f.h \ /usr/local/include/../pkg/src/f.h;
Note that the manual VPATH
search did not cause any problems here;
however this command installs f.h in an incorrect directory.
Trying to quote $(HEADERS)
in some way, as we did for
foo.c
a few makefiles ago, does not help:
install-HEADERS: $(HEADERS) headers='$(HEADERS)'; \ for i in $$headers; do \ $(INSTALL) -m 644 \ `test -f $$i || echo $(VPATH)/`$$i \ $(DESTDIR)$(includedir)/$$i; \ done
Now, headers='$(HEADERS)'
macro-expands to:
headers='f.h g.h h.h'
but g.h
is still a plain word. (As an aside, the idiom
headers='$(HEADERS)'; for i in $$headers;
is a good
idea if $(HEADERS)
can be empty, because some shells diagnose a
syntax error on for i in;
.)
One workaround is to strip this unwanted ../pkg/src/ prefix manually:
VPATH = ../pkg/src HEADERS = f.h g.h h.h install-HEADERS: $(HEADERS) headers='$(HEADERS)'; \ for i in $$headers; do \ i=`expr "$$i" : '$(VPATH)/\(.*\)'`; $(INSTALL) -m 644 \ `test -f $$i || echo $(VPATH)/`$$i \ $(DESTDIR)$(includedir)/$$i; \ done
Automake does something similar. However the above hack works only if
the files listed in HEADERS
are in the current directory or a
subdirectory; they should not be in an enclosing directory. If we had
HEADERS = ../f.h
, the above fragment would fail in a VPATH
build with Tru64 make
. The reason is that not only does
Tru64 make
rewrite dependencies, but it also simplifies
them. Hence ../f.h
becomes ../pkg/f.h
instead of
../pkg/src/../f.h
. This obviously defeats any attempt to strip
a leading ../pkg/src/ component.
The following example makes the behavior of Tru64 make
more apparent.
$ cat Makefile VPATH = sub all: ../foo echo ../foo $ ls Makefile foo $ make echo foo foo
Dependency ../foo was found in sub/../foo, but Tru64
make
simplified it as foo. (Note that the sub/
directory does not even exist, this just means that the simplification
occurred before the file was checked for.)
For the record here is how SunOS 4 make
behaves on this
example.
$ make make: Fatal error: Don't know how to make target `../foo' $ mkdir sub $ make echo sub/../foo sub/../foo
Next: Make Target Lookup, Previous: Automatic Rule Rewriting, Up: VPATH
and Make [Contents][Index]
make
Creates Prerequisite Directories MagicallyWhen a prerequisite is a subdirectory of VPATH
, Tru64
make
creates it in the current directory.
$ mkdir -p foo/bar build $ cd build $ cat >Makefile <<END VPATH = .. all: foo/bar END $ make mkdir foo mkdir foo/bar
This can yield unexpected results if a rule uses a manual VPATH
search as presented before.
VPATH = .. all : foo/bar command `test -d foo/bar || echo ../`foo/bar
The above command
is run on the empty foo/bar
directory that was created in the current directory.
Previous: Tru64 make
Creates Prerequisite Directories Magically, Up: VPATH
and Make [Contents][Index]
GNU make
uses a complex algorithm to decide when it
should use files found via a VPATH
search. See How Directory Searches are Performed in The GNU Make
Manual.
If a target needs to be rebuilt, GNU make
discards the
file name found during the VPATH
search for this target, and
builds the file locally using the file name given in the makefile.
If a target does not need to be rebuilt, GNU make
uses the
file name found during the VPATH
search.
Other make
implementations, like NetBSD make
, are
easier to describe: the file name found during the VPATH
search
is used whether the target needs to be rebuilt or not. Therefore
new files are created locally, but existing files are updated at their
VPATH
location.
OpenBSD and FreeBSD make
, however,
never perform a
VPATH
search for a dependency that has an explicit rule.
This is extremely annoying.
When attempting a VPATH
build for an autoconfiscated package
(e.g., mkdir build && cd build && ../configure
), this means
GNU
make
builds everything locally in the build
directory, while BSD make
builds new files locally and
updates existing files in the source directory.
$ cat Makefile VPATH = .. all: foo.x bar.x foo.x bar.x: newer.x @echo Building $@ $ touch ../bar.x $ touch ../newer.x $ make # GNU make Building foo.x Building bar.x $ pmake # NetBSD make Building foo.x Building ../bar.x $ fmake # FreeBSD make, OpenBSD make Building foo.x Building bar.x $ tmake # Tru64 make Building foo.x Building bar.x $ touch ../bar.x $ make # GNU make Building foo.x $ pmake # NetBSD make Building foo.x $ fmake # FreeBSD make, OpenBSD make Building foo.x Building bar.x $ tmake # Tru64 make Building foo.x Building bar.x
Note how NetBSD make
updates ../bar.x in its
VPATH location, and how FreeBSD, OpenBSD, and Tru64
make
always
update bar.x, even when ../bar.x is up to date.
Another point worth mentioning is that once GNU make
has
decided to ignore a VPATH
file name (e.g., it ignored
../bar.x in the above example) it continues to ignore it when
the target occurs as a prerequisite of another rule.
The following example shows that GNU make
does not look up
bar.x in VPATH
before performing the .x.y
rule,
because it ignored the VPATH
result of bar.x while running
the bar.x: newer.x
rule.
$ cat Makefile VPATH = .. all: bar.y bar.x: newer.x @echo Building $@ .SUFFIXES: .x .y .x.y: cp $< $@ $ touch ../bar.x $ touch ../newer.x $ make # GNU make Building bar.x cp bar.x bar.y cp: cannot stat `bar.x': No such file or directory make: *** [bar.y] Error 1 $ pmake # NetBSD make Building ../bar.x cp ../bar.x bar.y $ rm bar.y $ fmake # FreeBSD make, OpenBSD make echo Building bar.x cp bar.x bar.y cp: cannot stat `bar.x': No such file or directory *** Error code 1 $ tmake # Tru64 make Building bar.x cp: bar.x: No such file or directory *** Exit 1
Note that if you drop away the command from the bar.x: newer.x
rule, GNU make
magically starts to work: it
knows that bar.x
hasn’t been updated, therefore it doesn’t
discard the result from VPATH
(../bar.x) in succeeding
uses. Tru64 also works, but FreeBSD and OpenBSD
still don’t.
$ cat Makefile VPATH = .. all: bar.y bar.x: newer.x .SUFFIXES: .x .y .x.y: cp $< $@ $ touch ../bar.x $ touch ../newer.x $ make # GNU make cp ../bar.x bar.y $ rm bar.y $ pmake # NetBSD make cp ../bar.x bar.y $ rm bar.y $ fmake # FreeBSD make, OpenBSD make cp bar.x bar.y cp: cannot stat `bar.x': No such file or directory *** Error code 1 $ tmake # Tru64 make cp ../bar.x bar.y
It seems the sole solution that would please every make
implementation is to never rely on VPATH
searches for targets.
In other words, VPATH
should be reserved to sources that are not built.
Next: Timestamp Resolution and Make, Previous: VPATH
and Make, Up: Portable Make Programming [Contents][Index]
A Single Suffix Rule is basically a usual suffix (inference) rule (‘.from.to:’), but which destination suffix is empty (‘.from:’).
Separated dependencies simply refers to listing the prerequisite of a target, without defining a rule. Usually one can list on the one hand side, the rules, and on the other hand side, the dependencies.
Solaris make
does not support separated dependencies for
targets defined by single suffix rules:
$ cat Makefile .SUFFIXES: .in foo: foo.in .in: cp $< $@ $ touch foo.in $ make $ ls Makefile foo.in
while GNU Make does:
$ gmake cp foo.in foo $ ls Makefile foo foo.in
Note it works without the ‘foo: foo.in’ dependency.
$ cat Makefile .SUFFIXES: .in .in: cp $< $@ $ make foo cp foo.in foo
and it works with double suffix inference rules:
$ cat Makefile foo.out: foo.in .SUFFIXES: .in .out .in.out: cp $< $@ $ make cp foo.in foo.out
As a result, in such a case, you have to write target rules.
Previous: Single Suffix Rules and Separated Dependencies, Up: Portable Make Programming [Contents][Index]
Traditionally, file timestamps had 1-second resolution, and
make
used those timestamps to determine whether one file was
newer than the other. However, many modern file systems have
timestamps with 1-nanosecond resolution. Some make
implementations look at the entire timestamp; others ignore the
fractional part, which can lead to incorrect results. Normally this
is not a problem, but in some extreme cases you may need to use tricks
like ‘sleep 1’ to work around timestamp truncation bugs.
Commands like ‘cp -p’ and ‘touch -r’ typically do not copy file timestamps to their full resolutions (see Limitations of Usual Tools). Hence you should be wary of rules like this:
dest: src cp -p src dest
as dest often appears to be older than src after the
timestamp is truncated, and this can cause make
to do
needless rework the next time it is invoked. To work around this
problem, you can use a timestamp file, e.g.:
dest-stamp: src cp -p src dest date >dest-stamp
Apart from timestamp resolution, there are also differences in handling
equal timestamps. HP-UX make
updates targets if it has the
same timestamp as one of its prerequisites, in violation of Posix rules.
This can cause spurious rebuilds for repeated runs of make
.
This in turn can cause make
to fail if it tries to rebuild
generated files in a possibly read-only source tree with tools not
present on the end-user machine. Use GNU make
instead.
Next: Manual Configuration, Previous: Portable Make Programming, Up: Autoconf [Contents][Index]
C and C++ programs often use low-level features of the underlying system, and therefore are often more difficult to make portable to other platforms.
Several standards have been developed to help make your programs more portable. If you write programs with these standards in mind, you can have greater confidence that your programs work on a wide variety of systems. Language Standards Supported by GCC for a list of C-related standards. Many programs also assume the Posix standard.
Some old code is written to be portable to K&R C, which predates any C standard. K&R C compilers are no longer of practical interest, though, and the rest of section assumes at least C89, the first C standard.
Program portability is a huge topic, and this section can only briefly introduce common pitfalls. See Portability between System Types in The GNU Coding Standards, for more information.
Next: Integer Overflow, Up: Portable C and C++ Programming [Contents][Index]
Autoconf tests and ordinary programs often need to test what is allowed on a system, and therefore they may need to deliberately exceed the boundaries of what the standards allow, if only to see whether an optional feature is present. When you write such a program, you should keep in mind the difference between constraints, unspecified behavior, and undefined behavior.
In C, a constraint is a rule that the compiler must enforce. An example constraint is that C programs must not declare a bit-field with negative width. Tests can therefore reliably assume that programs with negative-width bit-fields are rejected by a compiler that conforms to the standard.
Unspecified behavior is valid behavior, where the standard allows multiple possibilities. For example, the order of evaluation of function arguments is unspecified. Some unspecified behavior is implementation-defined, i.e., documented by the implementation, but since Autoconf tests cannot read the documentation they cannot distinguish between implementation-defined and other unspecified behavior. It is common for Autoconf tests to probe implementations to determine otherwise-unspecified behavior.
Undefined behavior is invalid behavior, where the standard allows the implementation to do anything it pleases. For example, dereferencing a null pointer leads to undefined behavior. If possible, test programs should avoid undefined behavior, since a program with undefined behavior might succeed on a test that should fail.
The above rules apply to programs that are intended to conform to the standard. However, strictly-conforming programs are quite rare, since the standards are so limiting. A major goal of Autoconf is to support programs that use implementation features not described by the standard, and it is fairly common for test programs to violate the above rules, if the programs work well enough in practice.
Next: Preprocessor Arithmetic, Previous: Varieties of Unportability, Up: Portable C and C++ Programming [Contents][Index]
In practice many portable C programs assume that signed integer overflow wraps around reliably using two’s complement arithmetic. Yet the C standard says that program behavior is undefined on overflow, and in a few cases C programs do not work on some modern implementations because their overflows do not wrap around as their authors expected. Conversely, in signed integer remainder, the C standard requires overflow behavior that is commonly not implemented.
In languages like C, unsigned integer overflow reliably wraps around;
e.g., UINT_MAX + 1
yields zero.
This is guaranteed by the C standard and is
portable in practice, unless you specify aggressive,
nonstandard optimization options
suitable only for special applications.
In contrast, the C standard says that signed integer overflow leads to undefined behavior where a program can do anything, including dumping core or overrunning a buffer. The misbehavior can even precede the overflow. Such an overflow can occur during addition, subtraction, multiplication, division, and left shift.
Despite this requirement of the standard, many C programs and Autoconf tests assume that signed integer overflow silently wraps around modulo a power of two, using two’s complement arithmetic, so long as you cast the resulting value to a signed integer type or store it into a signed integer variable. If you use conservative optimization flags, such programs are generally portable to the vast majority of modern platforms, with a few exceptions discussed later.
For historical reasons the C standard also allows implementations with ones’ complement or signed magnitude arithmetic, but it is safe to assume two’s complement nowadays.
Also, overflow can occur when converting an out-of-range value to a signed integer type. Here a standard implementation must define what happens, but this might include raising an exception. In practice all known implementations support silent wraparound in this case, so you need not worry about other possibilities.
Next: Optimizations That Break Wraparound Arithmetic, Previous: Basics of Integer Overflow, Up: Integer Overflow [Contents][Index]
There has long been a tension between what the C standard requires for signed integer overflow, and what C programs commonly assume. The standard allows aggressive optimizations based on assumptions that overflow never occurs, but many practical C programs rely on overflow wrapping around. These programs do not conform to the standard, but they commonly work in practice because compiler writers are understandably reluctant to implement optimizations that would break many programs, unless perhaps a user specifies aggressive optimization.
The C Standard says that if a program has signed integer overflow its behavior is undefined, and the undefined behavior can even precede the overflow. To take an extreme example:
if (password == expected_password) allow_superuser_privileges (); else if (counter++ == INT_MAX) abort (); else printf ("%d password mismatches\n", counter);
If the int
variable counter
equals INT_MAX
,
counter++
must overflow and the behavior is undefined, so the C
standard allows the compiler to optimize away the test against
INT_MAX
and the abort
call.
Worse, if an earlier bug in the program lets the compiler deduce that
counter == INT_MAX
or that counter
previously overflowed,
the C standard allows the compiler to optimize away the password test
and generate code that allows superuser privileges unconditionally.
Despite this requirement by the standard, it has long been common for C code to assume wraparound arithmetic after signed overflow, and all known practical C implementations support some C idioms that assume wraparound signed arithmetic, even if the idioms do not conform strictly to the standard. If your code looks like the following examples it will almost surely work with real-world compilers.
Here is an example derived from the 7th Edition Unix implementation of
atoi
(1979-01-10):
char *p; int f, n; … while (*p >= '0' && *p <= '9') n = n * 10 + *p++ - '0'; return (f ? -n : n);
Even if the input string is in range, on most modern machines this has
signed overflow when computing the most negative integer (the -n
overflows) or a value near an extreme integer (the first +
overflows).
Here is another example, derived from the 7th Edition implementation of
rand
(1979-01-10). Here the programmer expects both
multiplication and addition to wrap on overflow:
static long int randx = 1; … randx = randx *<PHONE_NUMBER> + 12345; return (randx >> 16) & 077777;
In the following example, derived from the GNU C Library 2.5
implementation of mktime
(2006-09-09), the code assumes
wraparound arithmetic in +
to detect signed overflow:
time_t t, t1, t2; int sec_requested, sec_adjustment; … t1 = t + sec_requested; t2 = t1 + sec_adjustment; if (((t1 < t) != (sec_requested < 0)) | ((t2 < t1) != (sec_adjustment < 0))) return -1;
If your code looks like these examples, it is probably safe even though it does not strictly conform to the C standard. This might lead one to believe that one can generally assume wraparound on overflow, but that is not always true, as can be seen in the next section.
Next: Practical Advice for Signed Overflow Issues, Previous: Examples of Code Assuming Wraparound Overflow, Up: Integer Overflow [Contents][Index]
Compilers sometimes generate code that is incompatible with wraparound
integer arithmetic. A simple example is an algebraic simplification: a
compiler might translate (i * 2000) / 1000
to i * 2
because it assumes that i * 2000
does not overflow. The
translation is not equivalent to the original when overflow occurs:
e.g., in the typical case of 32-bit signed two’s complement wraparound
int
, if i
has type int
and value 1073742
,
the original expression returns -2147483 but the optimized
version returns the mathematically correct value 2147484.
More subtly, loop induction optimizations often exploit the undefined
behavior of signed overflow. Consider the following contrived function
sumc
:
int sumc (int lo, int hi) { int sum = 0; int i; for (i = lo; i <= hi; i++) sum ^= i * 53; return sum; }
To avoid multiplying by 53 each time through the loop, an optimizing
compiler might internally transform sumc
to the equivalent of the
following:
int transformed_sumc (int lo, int hi) { int sum = 0; int hic = hi * 53; int ic; for (ic = lo * 53; ic <= hic; ic += 53) sum ^= ic; return sum; }
This transformation is allowed by the C standard, but it is invalid for
wraparound arithmetic when INT_MAX / 53 < hi
, because then the
overflow in computing expressions like hi * 53
can cause the
expression i <= hi
to yield a different value from the
transformed expression ic <= hic
.
For this reason, compilers that use loop induction and similar
techniques often do not support reliable wraparound arithmetic when a
loop induction variable like ic
is involved. Since loop
induction variables are generated by the compiler, and are not visible
in the source code, it is not always trivial to say whether the problem
affects your code.
Hardly any code actually depends on wraparound arithmetic in cases like these, so in practice these loop induction optimizations are almost always useful. However, edge cases in this area can cause problems. For example:
int j; for (j = 1; 0 < j; j *= 2) test (j);
Here, the loop attempts to iterate through all powers of 2 that
int
can represent, but the C standard allows a compiler to
optimize away the comparison and generate an infinite loop,
under the argument that behavior is undefined on overflow. As of this
writing this optimization is not done by any production version of
GCC with -O2, but it might be performed by other
compilers, or by more aggressive GCC optimization options,
and the GCC developers have not decided whether it will
continue to work with GCC and -O2.
Next: Signed Integer Division and Integer Overflow, Previous: Optimizations That Break Wraparound Arithmetic, Up: Integer Overflow [Contents][Index]
Ideally the safest approach is to avoid signed integer overflow entirely. For example, instead of multiplying two signed integers, you can convert them to unsigned integers, multiply the unsigned values, then test whether the result is in signed range.
Rewriting code in this way will be inconvenient, though, particularly if
the signed values might be negative. Also, it may hurt
performance. Using unsigned arithmetic to check for overflow is
particularly painful to do portably and efficiently when dealing with an
integer type like uid_t
whose width and signedness vary from
platform to platform.
Furthermore, many C applications pervasively assume wraparound behavior and typically it is not easy to find and remove all these assumptions. Hence it is often useful to maintain nonstandard code that assumes wraparound on overflow, instead of rewriting the code. The rest of this section attempts to give practical advice for this situation.
If your code wants to detect signed integer overflow in sum = a +
b
, it is generally safe to use an expression like (sum < a) != (b
< 0)
.
If your code uses a signed loop index, make sure that the index cannot overflow, along with all signed expressions derived from the index. Here is a contrived example of problematic code with two instances of overflow.
for (i = INT_MAX - 10; i <= INT_MAX; i++) if (i + 1 < 0) { report_overflow (); break; }
Because of the two overflows, a compiler might optimize away or transform the two comparisons in a way that is incompatible with the wraparound assumption.
If your code uses an expression like (i * 2000) / 1000
and you
actually want the multiplication to wrap around on overflow, use
unsigned arithmetic
to do it, e.g., ((int) (i * 2000u)) / 1000
.
If your code assumes wraparound behavior and you want to insulate it against any GCC optimizations that would fail to support that behavior, you should use GCC’s -fwrapv option, which causes signed overflow to wrap around reliably (except for division and remainder, as discussed in the next section).
If you need to port to platforms where signed integer overflow does not reliably wrap around (e.g., due to hardware overflow checking, or to highly aggressive optimizations), you should consider debugging with GCC’s -ftrapv option, which causes signed overflow to raise an exception.
Previous: Practical Advice for Signed Overflow Issues, Up: Integer Overflow [Contents][Index]
Overflow in signed
integer division is not always harmless: for example, on CPUs of the
i386 family, dividing INT_MIN
by -1
yields a SIGFPE signal
which by default terminates the program. Worse, taking the remainder
of these two values typically yields the same signal on these CPUs,
even though the C standard requires INT_MIN % -1
to yield zero
because the expression does not overflow.
Next: Properties of Null Pointers, Previous: Integer Overflow, Up: Portable C and C++ Programming [Contents][Index]
In C99 and later, preprocessor arithmetic, used for #if
expressions, must
be evaluated as if all signed values are of type intmax_t
and all
unsigned values of type uintmax_t
. Many compilers are buggy in
this area, though. For example, as of 2007, Sun C mishandles #if
LLONG_MIN < 0
on a platform with 32-bit long int
and 64-bit
long long int
. Also, some older preprocessors mishandle
constants ending in LL
. To work around these problems, you can
compute the value of expressions like LONG_MAX < LLONG_MAX
at
configure
-time rather than at #if
-time.
Next: Buffer Overruns and Subscript Errors, Previous: Preprocessor Arithmetic, Up: Portable C and C++ Programming [Contents][Index]
Most modern hosts reliably fail when you attempt to dereference a null pointer.
On almost all modern hosts, null pointers use an all-bits-zero internal
representation, so you can reliably use memset
with 0 to set all
the pointers in an array to null values.
If p
is a null pointer to an object type, the C expression
p + 0
always evaluates to p
on modern hosts, even though
the standard says that it has undefined behavior.
Next: Volatile Objects, Previous: Properties of Null Pointers, Up: Portable C and C++ Programming [Contents][Index]
Buffer overruns and subscript errors are the most common dangerous errors in C programs. They result in undefined behavior because storing outside an array typically modifies storage that is used by some other object, and most modern systems lack runtime checks to catch these errors. Programs should not rely on buffer overruns being caught.
There is one exception to the usual rule that a portable program cannot
address outside an array. In C, it is valid to compute the address just
past an object, e.g., &a[N]
where a
has N
elements,
so long as you do not dereference the resulting pointer. But it is not
valid to compute the address just before an object, e.g., &a[-1]
;
nor is it valid to compute two past the end, e.g., &a[N+1]
. On
most platforms &a[-1] < &a[0] && &a[N] < &a[N+1]
, but this is not
reliable in general, and it is usually easy enough to avoid the
potential portability problem, e.g., by allocating an extra unused array
element at the start or end.
Valgrind can catch many overruns. GCC users might also consider using the -fsanitize= options to catch overruns. See Program Instrumentation Options in Using the GNU Compiler Collection (GCC).
Buffer overruns are usually caused by off-by-one errors, but there are more subtle ways to get them.
Using int
values to index into an array or compute array sizes
causes problems on typical 64-bit hosts where an array index might
be 2^{31} or larger. Index values of type size_t
avoid this
problem, but cannot be negative. Index values of type ptrdiff_t
are signed, and are wide enough in practice.
If you add or multiply two numbers to calculate an array size, e.g.,
malloc (x * sizeof y + z)
, havoc ensues if the addition or
multiplication overflows.
Many implementations of the alloca
function silently misbehave
and can generate buffer overflows if given sizes that are too large.
The size limits are implementation dependent, but are at least 4000
bytes on all platforms that we know about.
The standard functions asctime
, asctime_r
, ctime
,
ctime_r
, and gets
are prone to buffer overflows, and
portable code should not use them unless the inputs are known to be
within certain limits. The time-related functions can overflow their
buffers if given timestamps out of range (e.g., a year less than -999
or greater than 9999). Time-related buffer overflows cannot happen with
recent-enough versions of the GNU C library, but are possible
with other
implementations. The gets
function is the worst, since it almost
invariably overflows its buffer when presented with an input line larger
than the buffer.
Next: Floating Point Portability, Previous: Buffer Overruns and Subscript Errors, Up: Portable C and C++ Programming [Contents][Index]
The keyword volatile
is often misunderstood in portable code.
Its use inhibits some memory-access optimizations, but programmers often
wish that it had a different meaning than it actually does.
volatile
was designed for code that accesses special objects like
memory-mapped device registers whose contents spontaneously change.
Such code is inherently low-level, and it is difficult to specify
portably what volatile
means in these cases. The C standard
says, “What constitutes an access to an object that has
volatile-qualified type is implementation-defined,” so in theory each
implementation is supposed to fill in the gap by documenting what
volatile
means for that implementation. In practice, though,
this documentation is usually absent or incomplete.
One area of confusion is the distinction between objects defined with volatile types, and volatile lvalues. From the C standard’s point of view, an object defined with a volatile type has externally visible behavior. You can think of such objects as having little oscilloscope probes attached to them, so that the user can observe some properties of accesses to them, just as the user can observe data written to output files. However, the standard does not make it clear whether users can observe accesses by volatile lvalues to ordinary objects. For example:
/* Declare and access a volatile object. Accesses to X are "visible" to users. */ static int volatile x; x = 1; /* Access two ordinary objects via a volatile lvalue. It's not clear whether accesses to *P are "visible". */ int y; int *z = malloc (sizeof (int)); int volatile *p; p = &y; *p = 1; p = z; *p = 1;
Programmers often wish that volatile
meant “Perform the memory
access here and now, without merging several memory accesses, without
changing the memory word size, and without reordering.” But the C
standard does not require this. For objects defined with a volatile
type, accesses must be done before the next sequence point; but
otherwise merging, reordering, and word-size change is allowed. Worse,
it is not clear from the standard whether volatile lvalues provide more
guarantees in general than nonvolatile lvalues, if the underlying
objects are ordinary.
Even when accessing objects defined with a volatile type,
the C standard allows only
extremely limited signal handlers: in C99 the behavior is undefined if a signal
handler reads any non-local object, or writes to any non-local object
whose type is not sig_atomic_t volatile
, or calls any standard
library function other than abort
, signal
, and
_Exit
. Hence C compilers need not worry about a signal handler
disturbing ordinary computation. C11 and Posix allow some additional
behavior in a portable signal handler, but are still quite restrictive.
Some C implementations allow memory-access optimizations within each
translation unit, such that actual behavior agrees with the behavior
required by the standard only when calling a function in some other
translation unit, and a signal handler acts like it was called from a
different translation unit. The C99 standard hints that in these
implementations, objects referred to by signal handlers “would require
explicit specification of volatile
storage, as well as other
implementation-defined restrictions.” But unfortunately even for this
special case these other restrictions are often not documented well.
This area was significantly changed in C11, and eventually implementations
will probably head in the C11 direction, but this will take some time.
See When is a Volatile Object Accessed? in Using the
GNU Compiler Collection (GCC), for some
restrictions imposed by GCC. See Defining Signal Handlers in The GNU C Library, for some
restrictions imposed by the GNU C library. Restrictions
differ on other platforms.
If possible, it is best to use a signal handler that fits within the limits imposed by the C and Posix standards.
If this is not practical, you can try the following rules of thumb. A
signal handler should access only volatile lvalues, preferably lvalues
that refer to objects defined with a volatile type, and should not
assume that the accessed objects have an internally consistent state
if they are larger than a machine word. Furthermore, installers
should employ compilers and compiler options that are commonly used
for building operating system kernels, because kernels often need more
from volatile
than the C Standard requires, and installers who
compile an application in a similar environment can sometimes benefit
from the extra constraints imposed by kernels on compilers.
Admittedly we are hand-waving somewhat here, as there are few
guarantees in this area; the rules of thumb may help to fix some bugs
but there is a good chance that they will not fix them all.
For volatile
, C++ has the same problems that C does.
Multithreaded applications have even more problems with volatile
,
but they are beyond the scope of this section.
The bottom line is that using volatile
typically hurts
performance but should not hurt correctness. In some cases its use
does help correctness, but these cases are often so poorly understood
that all too often adding volatile
to a data structure merely
alleviates some symptoms of a bug while not fixing the bug in general.
Next: Exiting Portably, Previous: Volatile Objects, Up: Portable C and C++ Programming [Contents][Index]
Almost all modern systems use IEEE-754 floating point, and it is safe to assume IEEE-754 in most portable code these days. For more information, please see David Goldberg’s classic paper What Every Computer Scientist Should Know About Floating-Point Arithmetic.
Previous: Floating Point Portability, Up: Portable C and C++ Programming [Contents][Index]
A C or C++ program can exit with status N by returning
N from the main
function. Portable programs are supposed
to exit either with status 0 or EXIT_SUCCESS
to succeed, or with
status EXIT_FAILURE
to fail, but in practice it is portable to
fail by exiting with status 1, and test programs that assume Posix can
fail by exiting with status values from 1 through 255. Programs on
SunOS 2.0 (1985) through 3.5.2 (1988) incorrectly exited with zero
status when main
returned nonzero, but ancient systems like these
are no longer of practical concern.
A program can also exit with status N by passing N to the
exit
function, and a program can fail by calling the abort
function. If a program is specialized to just some platforms, it can fail
by calling functions specific to those platforms, e.g., _exit
(Posix). However, like other functions, an exit
function should be declared, typically by including a header. For
example, if a C program calls exit
, it should include stdlib.h
either directly or via the default includes (see Default Includes).
A program can fail due to undefined behavior such as dereferencing a null pointer, but this is not recommended as undefined behavior allows an implementation to do whatever it pleases and this includes exiting successfully.
Next: Site Configuration, Previous: Portable C and C++ Programming, Up: Autoconf [Contents][Index]
A few kinds of features can’t be guessed automatically by running test programs. For example, the details of the object-file format, or special options that need to be passed to the compiler or linker. Autoconf provides a uniform method for handling unguessable features, by giving each operating system a canonical system type, also known as a canonical name or target triplet.
If you use any of the macros described in this chapter, you must
distribute the helper scripts config.guess
and
config.sub
along with your source code. Some Autoconf macros
use these macros internally, so you may need to distribute these scripts
even if you do not use any of these macros yourself. See Configure Input: Source Code, Macros, and Auxiliary Files, for
information about the AC_CONFIG_AUX_DIR
macro which you can use
to control in which directory configure
looks for helper
scripts, and where to get the scripts from.
Next: Getting the Canonical System Type, Up: Manual Configuration [Contents][Index]
Autoconf-generated
configure
scripts can make decisions based on a canonical name
for the system type, or target triplet, which has the form:
‘cpu-vendor-os’, where os can be
‘system’ or ‘kernel-system’
configure
can usually guess the canonical name for the type of
system it’s running on. To do so it runs a script called
config.guess
, which infers the name using the uname
command or symbols predefined by the C preprocessor.
Alternately, the user can specify the system type with command line
arguments to configure
(see Specifying the System Type. Doing so is
necessary when
cross-compiling. In the most complex case of cross-compiling, three
system types are involved. The options to specify them are:
the type of system on which the package is being configured and
compiled. It defaults to the result of running config.guess
.
Specifying a build-type that differs from host-type enables
cross-compilation mode.
the type of system on which the package runs. By default it is the
same as the build machine. The tools that get used to build and
manipulate binaries will, by default, all be prefixed with
host-type-
, such as host-type-gcc
,
host-type-g++
, host-type-ar
, and
host-type-nm
. If the binaries produced by these tools can
be executed by the build system, the configure script will make use of
it in AC_RUN_IFELSE
invocations; otherwise, cross-compilation
mode is enabled. Specifying a host-type that differs
from build-type, when build-type was also explicitly
specified, equally enables cross-compilation mode.
the type of system for which any compiler tools in the package produce code (rarely needed). By default, it is the same as host.
If you mean to override the result of config.guess
but
still produce binaries for the build machine, use --build,
not --host.
So, for example, to produce binaries for 64-bit MinGW, use a command like this:
./configure --host=x86_64-w64-mingw64
If your system has the ability to execute MinGW binaries but you don’t want to make use of this feature and instead prefer cross-compilation guesses, use a command like this:
./configure --build=x86_64-pc-linux-gnu --host=x86_64-w64-mingw64
Note that if you do not specify --host, configure
fails if it can’t run the code generated by the specified compiler. For
example, configuring as follows fails:
./configure CC=x86_64-w64-mingw64-gcc
When cross-compiling, configure
will warn about any tools
(compilers, linkers, assemblers) whose name is not prefixed with the
host type. This is an aid to users performing cross-compilation.
Continuing the example above, if a cross-compiler named cc
is
used with a native pkg-config
, then libraries found by
pkg-config
will likely cause subtle build failures; but using
the names x86_64-w64-mingw64-gcc
and
x86_64-w64-mingw64-pkg-config
avoids any confusion. Avoiding the warning is as simple as creating the
correct symlinks naming the cross tools.
configure
recognizes short aliases for many system types; for
example, ‘decstation’ can be used instead of
‘mips-dec-ultrix4.2’. configure
runs a script called
config.sub
to canonicalize system type aliases.
This section deliberately omits the description of the obsolete interface; see Hosts and Cross-Compilation.
Next: Using the System Type, Previous: Specifying target triplets, Up: Manual Configuration [Contents][Index]
The following macros make the system type available to configure
scripts.
The variables ‘build_alias’, ‘host_alias’, and
‘target_alias’ are always exactly the arguments of --build,
--host, and --target; in particular, they are left empty
if the user did not use them, even if the corresponding
AC_CANONICAL
macro was run. Any configure script may use these
variables anywhere. These are the variables that should be used when in
interaction with the user.
If you need to recognize some special environments based on their system type, run the following macros to get canonical system names. These variables are not set before the macro call.
Compute the canonical build-system type variable, build
, and its
three individual parts build_cpu
, build_vendor
, and
build_os
.
If --build was specified, then build
is the
canonicalization of build_alias
by config.sub
,
otherwise it is determined by the shell script config.guess
.
Compute the canonical host-system type variable, host
, and its
three individual parts host_cpu
, host_vendor
, and
host_os
.
If --host was specified, then host
is the
canonicalization of host_alias
by config.sub
,
otherwise it defaults to build
.
Compute the canonical target-system type variable, target
, and its
three individual parts target_cpu
, target_vendor
, and
target_os
.
If --target was specified, then target
is the
canonicalization of target_alias
by config.sub
,
otherwise it defaults to host
.
Note that there can be artifacts due to the backward compatibility code. See Hosts and Cross-Compilation, for more.
Previous: Getting the Canonical System Type, Up: Manual Configuration [Contents][Index]
In configure.ac the system type is generally used by one or more
case
statements to select system-specifics. Shell wildcards can
be used to match a group of system types.
For example, an extra assembler code object file could be chosen, giving
access to a CPU cycle counter register. $(CYCLE_OBJ)
in the
following would be used in a makefile to add the object to a
program or library.
AS_CASE([$host], [alpha*-*-*], [CYCLE_OBJ=rpcc.o], [i?86-*-*], [CYCLE_OBJ=rdtsc.o], [CYCLE_OBJ=""] ) AC_SUBST([CYCLE_OBJ])
AC_CONFIG_LINKS
(see Creating Configuration Links) is another good way
to select variant source files, for example optimized code for some
CPUs. The configured CPU type doesn’t always indicate exact CPU types,
so some runtime capability checks may be necessary too.
case $host in alpha*-*-*) AC_CONFIG_LINKS([dither.c:alpha/dither.c]) ;; powerpc*-*-*) AC_CONFIG_LINKS([dither.c:powerpc/dither.c]) ;; *-*-*) AC_CONFIG_LINKS([dither.c:generic/dither.c]) ;; esac
The host system type can also be used to find cross-compilation tools
with AC_CHECK_TOOL
(see Generic Program and File Checks).
The above examples all show ‘$host’, since this is where the code is going to run. Only rarely is it necessary to test ‘$build’ (which is where the build is being done).
Whenever you’re tempted to use ‘$host’ it’s worth considering whether some sort of probe would be better. New system types come along periodically or previously missing features are added. Well-written probes can adapt themselves to such things, but hard-coded lists of names can’t. Here are some guidelines,
‘$target’ is for use by a package creating a compiler or similar. For ordinary packages it’s meaningless and should not be used. It indicates what the created compiler should generate code for, if it can cross-compile. ‘$target’ generally selects various hard-coded CPU and system conventions, since usually the compiler or tools under construction themselves determine how the target works.
Next: Running configure
Scripts, Previous: Manual Configuration, Up: Autoconf [Contents][Index]
configure
scripts support several kinds of local configuration
decisions. There are ways for users to specify where external software
packages are, include or exclude optional features, install programs
under modified names, and set default values for configure
options.
configure
OptionsNext: Working With External Software, Up: Site Configuration [Contents][Index]
Users consult ‘configure --help’ to learn of configuration
decisions specific to your package. By default, configure
breaks this output into sections for each type of option; within each
section, help strings appear in the order configure.ac defines
them:
Optional Features: … --enable-bar include bar Optional Packages: … --with-foo use foo
Request an alternate --help format, in which options of all
types appear together, in the order defined. Call this macro before any
AC_ARG_ENABLE
or AC_ARG_WITH
.
Optional Features and Packages: … --enable-bar include bar --with-foo use foo
Next: Choosing Package Options, Previous: Controlling Help Output, Up: Site Configuration [Contents][Index]
Some packages require, or can optionally use, other software packages
that are already installed. The user can give configure
command line options to specify which such external software to use.
The options have one of these forms:
--with-package[=arg] --without-package
For example, --with-gnu-ld means work with the GNU linker instead of some other linker. --with-x means work with The X Window System.
The user can give an argument by following the package name with ‘=’ and the argument. Giving an argument of ‘no’ is for packages that are used by default; it says to not use the package. An argument that is neither ‘yes’ nor ‘no’ could include a name or number of a version of the other package, to specify more precisely which other package this program is supposed to work with. If no argument is given, it defaults to ‘yes’. --without-package is equivalent to --with-package=no.
Normally configure
scripts complain about
--with-package options that they do not support.
See Controlling Checking of configure
Options, for details, and for how to override the
defaults.
For each external software package that may be used, configure.ac
should call AC_ARG_WITH
to detect whether the configure
user asked to use it. Whether each package is used or not by default,
and which arguments are valid, is up to you.
If the user gave configure
the option --with-package
or --without-package, run shell commands
action-if-given. If neither option was given, run shell commands
action-if-not-given. The name package indicates another
software package that this program should work with. It should consist
only of alphanumeric characters, dashes, plus signs, and dots.
The option’s argument is available to the shell commands
action-if-given in the shell variable withval
, which is
actually just the value of the shell variable named
with_package
, with any non-alphanumeric characters in
package changed into ‘_’. You may use that variable instead,
if you wish.
Note that action-if-not-given is not expanded until the point that
AC_ARG_WITH
was expanded. If you need the value of
with_package
set to a default value by the time argument
parsing is completed, use m4_divert_text
to the DEFAULTS
diversion (see m4_divert_text) (if done as an argument to
AC_ARG_WITH
, also provide non-diverted text to avoid a shell
syntax error).
The argument help-string is a description of the option that looks like this:
--with-readline support fancy command line editing
help-string may be more than one line long, if more detail is
needed. Just make sure the columns line up in ‘configure
--help’. Avoid tabs in the help string. The easiest way to provide the
proper leading whitespace is to format your help-string with the macro
AS_HELP_STRING
(see Making Your Help Strings Look Pretty).
The following example shows how to use the AC_ARG_WITH
macro in
a common situation. You want to let the user decide whether to enable
support for an external library (e.g., the readline library); if the user
specified neither --with-readline nor --without-readline,
you want to enable support for readline only if the library is available
on the system.
AC_ARG_WITH([readline], [AS_HELP_STRING([--with-readline], [support fancy command line editing @<:@default=check@:>@])], [], [: m4_divert_text([DEFAULTS], [with_readline=check])]) LIBREADLINE= AS_IF([test "x$with_readline" != xno], [AC_CHECK_LIB([readline], [main], [AC_SUBST([LIBREADLINE], ["-lreadline -lncurses"]) AC_DEFINE([HAVE_LIBREADLINE], [1], [Define if you have libreadline]) ], [if test "x$with_readline" != xcheck; then AC_MSG_FAILURE( [--with-readline was given, but test for readline failed]) fi ], -lncurses)])
The next example shows how to use AC_ARG_WITH
to give the user the
possibility to enable support for the readline library, in case it is still
experimental and not well tested, and is therefore disabled by default.
AC_ARG_WITH([readline], [AS_HELP_STRING([--with-readline], [enable experimental support for readline])], [], [with_readline=no]) LIBREADLINE= AS_IF([test "x$with_readline" != xno], [AC_CHECK_LIB([readline], [main], [AC_SUBST([LIBREADLINE], ["-lreadline -lncurses"]) AC_DEFINE([HAVE_LIBREADLINE], [1], [Define if you have libreadline]) ], [AC_MSG_FAILURE( [--with-readline was given, but test for readline failed])], [-lncurses])])
The last example shows how to use AC_ARG_WITH
to give the user the
possibility to disable support for the readline library, given that it is
an important feature and that it should be enabled by default.
AC_ARG_WITH([readline], [AS_HELP_STRING([--without-readline], [disable support for readline])], [], [with_readline=yes]) LIBREADLINE= AS_IF([test "x$with_readline" != xno], [AC_CHECK_LIB([readline], [main], [AC_SUBST([LIBREADLINE], ["-lreadline -lncurses"]) AC_DEFINE([HAVE_LIBREADLINE], [1], [Define if you have libreadline]) ], [AC_MSG_FAILURE( [readline test failed (--without-readline to disable)])], [-lncurses])])
These three examples can be easily adapted to the case where
AC_ARG_ENABLE
should be preferred to AC_ARG_WITH
(see
Choosing Package Options).
Next: Making Your Help Strings Look Pretty, Previous: Working With External Software, Up: Site Configuration [Contents][Index]
If a software package has optional compile-time features, the user can
give configure
command line options to specify whether to
compile them. The options have one of these forms:
--enable-feature[=arg] --disable-feature
These options allow users to choose which optional features to build and install. --enable-feature options should never make a feature behave differently or cause one feature to replace another. They should only cause parts of the program to be built rather than left out.
The user can give an argument by following the feature name with ‘=’ and the argument. Giving an argument of ‘no’ requests that the feature not be made available. A feature with an argument looks like --enable-debug=stabs. If no argument is given, it defaults to ‘yes’. --disable-feature is equivalent to --enable-feature=no.
Normally configure
scripts complain about
--enable-package options that they do not support.
See Controlling Checking of configure
Options, for details, and for how to override the
defaults.
For each optional feature, configure.ac should call
AC_ARG_ENABLE
to detect whether the configure
user asked
to include it. Whether each feature is included or not by default, and
which arguments are valid, is up to you.
If the user gave configure
the option
--enable-feature or --disable-feature, run
shell commands action-if-given. If neither option was given, run
shell commands action-if-not-given. The name feature
indicates an optional user-level facility. It should consist only of
alphanumeric characters, dashes, plus signs, and dots.
The option’s argument is available to the shell commands
action-if-given in the shell variable enableval
, which is
actually just the value of the shell variable named
enable_feature
, with any non-alphanumeric characters in
feature changed into ‘_’. You may use that variable instead,
if you wish. The help-string argument is like that of
AC_ARG_WITH
(see Working With External Software).
Note that action-if-not-given is not expanded until the point that
AC_ARG_ENABLE
was expanded. If you need the value of
enable_feature
set to a default value by the time argument
parsing is completed, use m4_divert_text
to the DEFAULTS
diversion (see m4_divert_text) (if done as an argument to
AC_ARG_ENABLE
, also provide non-diverted text to avoid a shell
syntax error).
You should format your help-string with the macro
AS_HELP_STRING
(see Making Your Help Strings Look Pretty).
See the examples suggested with the definition of AC_ARG_WITH
(see Working With External Software) to get an idea of possible applications of
AC_ARG_ENABLE
.
Next: Controlling Checking of configure
Options, Previous: Choosing Package Options, Up: Site Configuration [Contents][Index]
Properly formatting the ‘help strings’ which are used in
AC_ARG_WITH
(see Working With External Software) and AC_ARG_ENABLE
(see Choosing Package Options) can be challenging. Specifically, you want
your own ‘help strings’ to line up in the appropriate columns of
‘configure --help’ just like the standard Autoconf ‘help
strings’ do. This is the purpose of the AS_HELP_STRING
macro.
Expands into a help string that looks pretty when the user executes
‘configure --help’. It is typically used in AC_ARG_WITH
(see Working With External Software) or AC_ARG_ENABLE
(see Choosing Package Options). The following example makes this clearer.
AC_ARG_WITH([foo], [AS_HELP_STRING([--with-foo], [use foo (default is no)])], [use_foo=$withval], [use_foo=no])
Then the last few lines of ‘configure --help’ appear like this:
--enable and --with options recognized: --with-foo use foo (default is no)
Macro expansion is performed on the first argument. However, the second
argument of AS_HELP_STRING
is treated as a whitespace separated
list of text to be reformatted, and is not subject to macro expansion.
Since it is not expanded, it should not be double quoted.
See The Autoconf Language, for a more detailed explanation.
The AS_HELP_STRING
macro is particularly helpful when the
left-hand-side and/or right-hand-side are composed of macro
arguments, as shown in the following example. Be aware that
left-hand-side may not expand to unbalanced quotes,
although quadrigraphs can be used.
AC_DEFUN([MY_ARG_WITH], [AC_ARG_WITH(m4_translit([[$1]], [_], [-]), [AS_HELP_STRING([--with-m4_translit([$1], [_], [-])], [use $1 (default is $2)])], [use_[]$1=$withval], [use_[]$1=$2])]) MY_ARG_WITH([a_b], [no])
Here, the last few lines of ‘configure --help’ will include:
--enable and --with options recognized: --with-a-b use a_b (default is no)
The parameters indent-column and wrap-column were introduced in Autoconf 2.62. Generally, they should not be specified; they exist for fine-tuning of the wrapping.
AS_HELP_STRING([--option], [description of option]) ⇒ --option description of option AS_HELP_STRING([--option], [description of option], [15], [30]) ⇒ --option description of ⇒ option
Next: Configuring Site Details, Previous: Making Your Help Strings Look Pretty, Up: Site Configuration [Contents][Index]
configure
OptionsThe configure
script checks its command-line options against a
list of known options, like --help or --config-cache.
An unknown option ordinarily indicates a mistake by the user and
configure
halts with an error. However, by default unknown
--with-package and --enable-feature
options elicit only a warning, to support configuring entire source
trees.
Source trees often contain multiple packages with a top-level
configure
script that uses the AC_CONFIG_SUBDIRS
macro
(see Configuring Other Packages in Subdirectories). Because the packages generally support
different --with-package and
--enable-feature options, the GNU Coding
Standards say they must accept unrecognized options without halting.
Even a warning message is undesirable here, so AC_CONFIG_SUBDIRS
automatically disables the warnings.
This default behavior may be modified in two ways. First, the installer
can invoke configure --disable-option-checking
to disable
these warnings, or invoke configure --enable-option-checking=fatal
options to turn them into fatal errors, respectively. Second, the
maintainer can use AC_DISABLE_OPTION_CHECKING
.
By default, disable warnings related to any unrecognized
--with-package or --enable-feature
options. This is implied by AC_CONFIG_SUBDIRS
.
The installer can override this behavior by passing
--enable-option-checking (enable warnings) or
--enable-option-checking=fatal (enable errors) to
configure
.
Next: Transforming Program Names When Installing, Previous: Controlling Checking of configure
Options, Up: Site Configuration [Contents][Index]
Some software packages require complex site-specific information. Some examples are host names to use for certain services, company names, and email addresses to contact. Since some configuration scripts generated by Metaconfig ask for such information interactively, people sometimes wonder how to get that information in Autoconf-generated configuration scripts, which aren’t interactive.
Such site configuration information should be put in a file that is
edited only by users, not by programs. The location of the file
can either be based on the prefix
variable, or be a standard
location such as the user’s home directory. It could even be specified
by an environment variable. The programs should examine that file at
runtime, rather than at compile time. Runtime configuration is more
convenient for users and makes the configuration process simpler than
getting the information while configuring. See Variables for Installation Directories in The GNU Coding
Standards, for more information on where to put data files.
Next: Setting Site Defaults, Previous: Configuring Site Details, Up: Site Configuration [Contents][Index]
Autoconf supports changing the names of programs when installing them.
In order to use these transformations, configure.ac must call the
macro AC_ARG_PROGRAM
.
Place in output variable program_transform_name
a sequence of
sed
commands for changing the names of installed programs.
If any of the options described below are given to configure
,
program names are transformed accordingly. Otherwise, if
AC_CANONICAL_TARGET
has been called and a --target value
is given, the target type followed by a dash is used as a prefix.
Otherwise, no program name transformation is done.
You can specify name transformations by giving configure
these
command line options:
prepend prefix to the names;
append suffix to the names;
perform sed
substitution expression on the names.
Next: Transformation Rules, Previous: Transformation Options, Up: Transforming Program Names When Installing [Contents][Index]
These transformations are useful with programs that can be part of a cross-compilation development environment. For example, a cross-assembler running on x86-64 configured with --target=aarch64-linux-gnu is normally installed as aarch64-linux-gnu-as, rather than as, which could be confused with a native x86-64 assembler.
You can force a program name to begin with g, if you don’t want
GNU programs installed on your system to shadow other programs with
the same name. For example, if you configure GNU diff
with
--program-prefix=g, then when you run ‘make install’ it is
installed as /usr/local/bin/gdiff.
As a more sophisticated example, you could use
--program-transform-name='s/^/g/; s/^gg/g/; s/^gless/less/'