If you look carefully at the above examples you will see one of the potential
dangers of using __trusted_cast: you are on your own to ensure that the
argument type and the result type match. In the above example, this is not
true because the field f1 in struct bar is SAFE while the field
f1 in struct foo is FSEQ!
If you want to prevent a pointer arithmetic operation from generating
sequence pointers, you can use the __trusted_add function:
int foo(int *p) {
int * q = __trusted_add(p, 4);
return *q;
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
You can use a __trusted_cast to cast an integer into a pointer. This
works as expected if the type of the resulting pointer is SAFE (as in the
example with boxedint earlier in this section). But if it is FSEQ or
SEQ then you will get exactly the same effect as if the
__trusted_cast was not there: you will obtain a pointer with null
metadata and thus unusable for memory dereference.
A better way to cast an integer (or a SAFE pointer into a SEQ or
FSEQ one) is to use the __mkptr built-in function. This function
takes as a second argument some other pointer whose metadata is used in
constructing the result:
int g[8];
int main() {
int * __SAFE pg = & g[2];
int * __SEQ sg = __mkptr(pg, g); // We know that the home area of pg and g
// are the same
int pg1 = (int) & g[3];
int * __SEQ sg1 = __mkptr(pg1, g);
return sg[1] + sg1[1];
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
Another useful built-in function is __mkptr_size. It allows you to
specify the size of the home area in which a pointer lives:
int g[8];
int main() {
int * __SAFE pg = & g[2];
// We know that there is are at least 2 more integers after pg
int * __SEQ sg = __mkptr_size(pg, 2 * sizeof(int));
// are the same
return sg[1];
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
There are other built-in functions that you can use to achieve various things
behind CCured's back. Those are mostly intended for use in wrappers for the
library functions (which you have to trust anyway). These are described in
Chapter 8 and declared in ccured.h.
7.4.2 Turning off curing
You can turn the curing off for a fragment of a source file, for a function,
or for a block statement.
You can use the cure pragma to turn curing off for a fragment of a source
file (in CCured pragmas can only appear at global scope and therefore you
cannot use this mechanism to turn curing off for part of the definition of a
global function):
int * g; // This is a pointer to several integers
// but we do not want to make it SEQ
#pragma ccured(off)
int foo() {
return g[2]; // CCured won't see this and will leave g SAFE
// But also CCured won't check this code
}
#pragma ccured(on)
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
Alternatively, you can add the nocure attribute to a function to tell
CCured to not cure this function:
int * g; // This is a pointer to several integers
// but we do not want to make it SEQ
// We must put the attribute in a prototype
int foo(void) __NOCURE;
int foo(void) {
return g[2]; // CCured won't see this and will leave g SAFE
// But also CCured won't check this code
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
At a finer-grained level, you can use the __NOCUREBLOCK attribute with
a block statement:
int * g; // This is a pointer to several integers
// but we do not want to make it SEQ
int foo(void) {
int res;
{ __NOCUREBLOCK
res = g[2]; // CCured won't see this and will leave g SAFE
}
return res;
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
In all of these cases, the CCured inferencer does not even look at the
non-cured portions of the code. However, CCured will at least change the
non-cured code to access the fat pointers properly. For example, in the
following example the global g is a sequence pointer. While CCured will
not complain about the unsafe cast to int **, it will make sure that at
least the proper component of g is used:
int * g; // This will be FSEQ
int ** foo(void) {
int res = g[2]; // Make g FSEQ
{ __NOCUREBLOCK
return (int **)g; // But not WILD
}
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
Finally, to avoid curing a whole source file (say trusted_foo.c), you can
use the –leavealone=trusted argument to CCured. All source files whose
names start with the given “leave alone” prefix, are not merged and are not
scanned by CCured at all. Instead they are compiled with gcc and linked in
the final executable.
C programs require linking with external libraries before they can be run.
Most often this is just the standard C library but occasionally your program
might want to use other libraries as well. Often you do not want to
process the library code with CCured, either because you do not have the
source code or because it would be inconvenient to do so. There are several
problems that arise when you link code that was cured with regular code:
-
Since CCured does not inspect the library code it is possible for your
program to be unsafe. This can happen even for correctly written libraries if
the client code passes the wrong arguments.
- Much more seriously, CCured alters the layout of some data structures to
insert the metadata for the pointers that they contain. This essentially means
that CCured alters the calling convention for functions whose parameter or
return types involve pointers. Linking cured code with a regular library
is a disaster waiting to happen. CCured detects at link time when this is a
problem but it is up to you to provide a solution, as explained in the rest
of this chapter.
Both of the above problems can be solved by interposing small wrapper
functions around calls to the external functions. Such a wrapper function can
perform the necessary checks on the arguments (thus addressing the first
problem above) and can also perform some conversions on the arguments and
result values before and after invoking the external function (thus addressing
the second problem above). However, to write a wrapper you must understand the
input-output behavior of the function (i.e., you must read the man-page
entry for that function). When writing wrappers you can use a number of
built-in functions described in Section 8.5.
Before we look at how one writes wrapper functions, we'll explain how CCured
detects the need for a wrapper and how it reports it to you. CCured changes
the name of all functions that it processes by adding a suffix describing the
kinds of the pointers involved in the prototype of the function. This process
is called name mangling and is described in Section 8.1. For example,
consider the strchr function from the standard library
(it returns NULL or a pointer
to the first occurrence of chr into the str argument):
char* strchr(char* str, int chr);
Most likely CCured will view code that uses the returned pointer as a
manipulating a pointer to a sequence,
and will require that the result of strchr be a FSEQ pointer. In the
cured code you will see the following prototype for strchr in that case:
fseqp_char strchr_fs(char* __SAFE str, int chr);
Note that CCured has added a suffix _fs to the name of the function to
say that its return value is a FSEQ pointer. When the code is passed to
gcc to link you will likely see the following error message:
main.c: undefined reference to `strchr_fs`
This is how you learn that the cured code requires a strchr with a
different calling convention than the one provided by the library. (If CCured
had left alone both the type and the name of strchr the linker would
not have complained but the program would have behaved incorrectly at
run-time.).
In the next section we describe in detail the name mangling algorithm, even
though it is probably better to use the browser to inspect the type of the
undefined mangled functions.
8.1 Name Mangling
There are two forms of mangling: shallow and deep. Shallow mangling
differentiates types without descending into structures. It is less verbose
and is used on all functions and variables defined in the code being cured.
Deep mangling does consider the types of the structure fields and is used for
imported (used but not defined) symbols. (The –shallowMangling command
line option causes all symbols to be mangled using shallow mangling.)
The mangling algorithm scans the type of a global and produces a suffix
that is appended to the name of the global.
-
The mangling of a scalar type is empty.
- The mangling of a function type consists of the mangling of the result
type followed by the mangling of the formal argument types in order.
(For vararg functions CCured also considers the fields of the
descriptor structure as if they were formal arguments in place of the
“...”. See Section 9.6 for details in vararg functions).
- The mangling of a pointer type consists of one letter that encodes the
kind of pointer, followed by the mangling of the base type, as
follows:
-
s for SAFE
- f for FSEQ
- q for SEQ
- i for INDEX
- w for WILD
- rs for RTTI (a SAFEpointer with RTTI information)
- rf for FSEQR (an FSEQpointer with RTTI information)
- rs for SEQR (a SEQpointer with RTTI information)
- v for a va_list in a vararg function.
- t for a tagged function. If you get a linker error on
“foo_t”, it probably means you are missing the declaration for
function “foo” (and as a result the inference algorithm made worst-case
assumptions about it).
- The mangling of an array is the mangling of the element type, preceded
by l (lowercase L) if the array has a length word.
- The mangling of a structure or union type is empty for shallow mangling.
For deep mangling it is c (for “composite”) followed by the mangling
of field types (only the widest field for a union) followed by a matching
_. As an exception, if the mangling of the field types is empty or
consisting only of s (structure whose fields are scalars or SAFE
pointers to scalars) then the mangling of the structure is empty. Note that
under deep mangling only the first occurrence of the structure type in a
global's type is mangled.
The mangling computed as above is appended to the global name, following a
_ character. As an exception, if the mangling is empty or contains only
s characters, then no mangling occurs (and no _ is inserted).
Here are a few of examples of mangling:
struct list {
void * __WILD data;
struct list * __SAFE next;
};
struct hash {
int count;
struct list * __FSEQ buckets;
};
struct list * __FSEQ getBuckets(struct hash * __SAFE);
// has deep mangling getBuckets_fcws_scf_
// has shallow mangling getBuckets_fs
struct twolists {
struct list * __SEQ one;
struct list * __SAFE two;
};
struct twolists * __SAFE getTwoLists(void);
// has deep mangling getTwoLists_scqcws_s_
// has shallow mangling getTwoLists
int main() {
printf("Mangling of struct list = %s\n", CCURED_MANGLING_OF(struct list));
printf("Mangling of getBuckets = %s\n", CCURED_MANGLING_OF(getBuckets));
}
Note that types are mangled recursively (you can have nested constructions
c..._ but a structure is mangled only once in a global's type.
The general rule is that the shallow mangling can be obtained from the deep
mangling be removing all c..._ sequences (including nested ones). If what
remains is a sequence of s's then the shallow mangling is empty.
CCured has a facility for inquiring about the mangling of a type or variable
at run-time, as shown at the end of the above code. You can use the macro
CCURED_MANGLING_OF applied to a type or variable (i.e. anything that can
be the argument of sizeof) to obtain a string literal that encodes the
mangling. Perhaps the most important use of this is to detect at run-time
whether a global is compatible with the library or not (its mangling should be
the empty string). We shall see how this is done later in this chapter. CCured
also defines the macro CCURED_HAS_EMPTY_MANGLING that yields 1 if its
parameter has empty mangling. This is equivalent to 0 == *
CCURED_MANGLING_OF but is a compile-time constant (after curing).
8.2 Writing Simple Wrappers
Returning to our strchr example, what do we do if the linker complains that
it needs the strchr_fs version of the function? The basic idea is that
we write the function strchr_fs that in turn calls strchr from the
library. The problem with this approach is that over time you might have to
write many versions of strchr (such as strchr_qs,
strchr_qq and many more). CCured provides a relatively simple mechanism
by which you write a wrapper specification and it generates all the required
versions automatically from your specification. Furthermore, the wrapper
specification looks just like a C function (along with some indication to
CCured that this is a wrapper):
extern char* strchr(char*, int); // Make sure you have a prototype for the
// function you are wrapping
extern void exit(int);
// Now tell CCured that strchr_wrapper is
// a wrapper for strchr
#pragma ccuredwrapper("strchr_wrapper", of("strchr"))
char* strchr_wrapper(char* str, int chr) // must match strchr's signature
{
char* result;
result = strchr(str, chr); // Call the underlying function.
return __mkptr_string(result); // result should be 0 or a string
}
void foo(char* s){
char* res = strchr(s, 'q');
// Taking the address of a wrapped function is handled, too
char * (*p_strchr)(char *, int) = & strchr;
if(res != (*p_strchr)(s, 'q')) exit(1);
res ++; // Make sure res is __FSEQ
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
Notice that we have used the __mkptr_string helper function that takes
in a standard null-terminated C string and produces any required fat pointer
to it. This is done using a simple trick: we let CCured mangle the helper
function name as needed and we provide in ccuredlib.c all the required
versions. Since the argument should always be a SAFE pointer the versions that
we provide are: __mkptr_string_fs, __mkptr_string_qs and, of
course, __mkptr_string, which is the identity function. These
functions produce the necessary metadata by looking for the null-termination.
Call the __mkptr_string function only on the result of trusted library
functions, or else bad things might happen (e.g., because the string
argument may not actually be null-terminated).
Now, if you click the “Browse” link for the above code fragment you will
see several things:
-
CCured creates a new instance of strchr_wrapper for each call site
or place where the address of strchr is taken, whose names differ in the
/*xxx*/ prefix. Each instance has its own body. In other words, all
wrappers are polymorphic (see Section 7.1) which means that the constraints
from one call site will not pollute other call-sites.
- CCured replaces all occurrences of strchr (either in call sites, or
where their address is taken with a distinct instance of the
strchr_wrapper). (Note, this means that you should not compare the
address of such functions.)
- Since res is required to be FSEQ the return type of the first
instance of the strchr wrapper must also have a FSEQ kind. But the
return type of the second instance is SAFE.
Now if you look at the CCured output for the above example you will see that
the code has two instances of the wrapper, one for each calling convention.
(If you had more calls sites for strchr CCured will coalesce all wrappers
that end up having the same mangling.)
It is very important to write the wrappers such that the actual call to the
underlying function has arguments and result values that are SAFE (i.e.,
values that use the standard C representation)! This
means two things:
-
You must “isolate” the arguments and result values of the actual call
from the corresponding elements of the wrapper. Otherwise, CCured will flow
the constraints of the latter straight to the prototype for the underlying
function, which ends up with a mangled name again.
Consider the following (wrong) version of wrapper:
extern char* strchr(char*, int);
#pragma ccuredwrapper("strchr_wrapper2", of("strchr"))
char* strchr_wrapper2(char* str, int chr)
{
char* result;
result = strchr(str, chr);
return result; // !! We return the result directly
}
void foo(char* s){
char* res = strchr(s, 'q');
res ++; // Force res to be __FSEQ
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
You get the following error message from CCured:
main.c:10: Error: strchr appears to be external
(it has a wrapper), yet it has a mangled name: strchr_fs.
Did you forget to use __ptrof and a version of __mkptr?
For more information, consult the online documentation on
"Writing Wrappers" chapter.
Click the “Browse” link for this code to see that strchr has a FSEQ
return type and that this happens because of the return instruction in the
strchr_wrapper. The solution is to always “isolate” the return values
using one of the __mkptr, __mkptr_size and __mkptr_string
(all documented in ccured.h), as we have done in the first
wrapper we wrote.
But even the supposedly good wrapper has similar problems. The argument
str is passed directly to strchr, so it is possible that some
constraints on that argument from the client program will flow to the
underlying strchr. To prevent this problem you should change the wrapper
as follows:
extern char* strchr(char*, int);
#pragma ccuredwrapper("strchr_wrapper3", of("strchr"))
char* strchr_wrapper3(char* str, int chr)
{
char* result;
result = strchr(__ptrof_nocheck(str), chr); // Isolate the argument str
return __mkptr_string(result);
}
void foo(char* s){
char* res = strchr(s, 'q');
res ++; // Force res to be __FSEQ
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
The __ptrof_nocheck function (described in ccured.h)
extracts the pointer value from a potentially fat pointer. As its
name says, it does no checks on that value. We'll describe other variants that
do some checking later on.
- Another way to flow un-SAFE pointers into the call to the underlying
function is to perform pointer arithmetic or casts in the wrapper code.
Consider this example of wrapper that does all sorts of bad things:
extern char* strchr(char*, int);
extern int strlen(char *);
#pragma ccuredwrapper("strchr_wrapper4", of("strchr"))
char* strchr_wrapper4(char* str, int chr)
{
char * __SAFE result;
char * __SAFE safestr = __ptrof_nocheck(str);
result ++; // Make it FSEQ! Bad
result = __trusted_add(result, 1); // Do this if you want to add
result = (int***)result; // Cast, make it WILD! Bad
result = (int***)__trusted_cast(result); // Do this if you want to cast
result = strchr(safestr, chr);
return __mkptr_string(result);
}
void foo(char* s){
char* res = strchr(s, 'q');
res ++; // Force res to be __FSEQ
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
If you need to do pointer arithmetic or casts you must do them behind CCured's
back (e.g. using __trusted_add or __trusted_cast). No new unsoundness
is introduced by calling potentially unsafe functions in the
wrapper, since the wrapper itself must be trusted. See Chapter 7 for
other methods to work behind CCured's back.
So far we have addressed the issue of compatibility with the library but it
is still possible for the client program to call strchr with the wrong
arguments, thus provoking a memory safety violation in the library itself. To
prevent this we add to the wrapper code that verifies that the input argument
is a valid string (all the characters up to and including the null-termination
are in the home area):
extern char* strchr(char*, int);
#pragma ccuredwrapper("strchr_wrapper5", of("strchr"))
char* strchr_wrapper5(char* str, int chr)
{
char* result;
result = strchr(
__stringof(str), // Check that str is a string
chr
);
return result; //Passing the return value directly
}
void foo(char* s){
char* res = strchr(s, 'q');
res ++; // Force res to be __FSEQ
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
The __stringof function first checks that str is a valid string and
then returns the pointer value (leaving the metadata behind). Alternatively,
the __ptrof function performs bounds checks on FSEQ and SEQ
pointers and null checks as well.
If you browse the above example you will observe that now the “s” argument
of the foo function is not SAFE anymore but is FSEQ. If you
investigate you find that this is required by the __string_of function
itself (in order to be able to perform safely the check that there is a
null-character before the end of the memory area!).
There are quite a few helper functions that you can use in the wrapper.
They are all documented in ccured.h.
We have not yet discussed the issue of where to put the wrappers that you
write. The rule is that the wrapper, along with the #pragma that declares it,
must appear somewhere in the project (CCured merges all files together so it
will eventually see it). You should put the pragma next to the wrapper
function because otherwise the CIL front-end might think that the wrapper is
not used and will remove it from the code!
To ensure that you have the wrapper available whenever you want to use the
library it is a good idea to put the wrapper in the header file that declares
the corresponding function (in that case we suggest you use “__inline
static” as declaration specifiers). We also suggest that you put the wrappers
at the end of such header files to ensure that you have the prototypes for the
wrapper functions already. We have done just that with the wrappers
that we wrote for the standard C library. You can find them in files in
cil/include directory. For example the stdio_wrappers.h file contains
the wrappers for the stdio.h standard header (We use a patched version of
stdio.h that includes stdio_wrappers at the end.) We suggest that you
look through those files to find more examples of wrappers.
8.3 Writing Complex Wrappers
For most wrappers the description from the previous section should be enough.
Some wrappers, however, are more complicated. We discuss in this section some
of these complications, along with the CCured mechanisms that address them,
for a function similar to the sendmsg function (defined in
<socket.h>). Here are the required declarations, along with a code
fragment that uses sendmsg in such a way that its msg_iov field
becomes FSEQ.:
int sendmsg(int fd, struct msghdr * msg, int flags);
struct msghdr {
void * msg_name; /* optional address */
int msg_namelen; /* size of address */
struct iovec * msg_iov; /* scatter/gather array */
int msg_iovlen; /* # elements in msg_iov */
void * msg_control; /* ancillary data, see below */
int msg_controllen; /* ancillary data buffer len */
int msg_flags; /* flags on received message */
};
struct iovec {
char* iov_base; /* The base address of the fragment */
int iov_len; /* Length in bytes of the fragment */
};
int foo(int fd, struct iovec *array, int array_len) {
struct msghdr msg = { 0, 0, array, array_len, 0, 0, 0};
// Make the msg_iov be FSEQ
struct iovec * foo = msg.msg_iov + 1;
return sendmsg(fd, &msg, 0);
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
Since our code does arithmetic on the msg_iov field, it becomes FSEQ
and the required mangling for sendmsg is sendmsg_scsfs (all the other
pointers in the prototype remain SAFE). Let us try the wrapper-writing
method from above (isolate the arguments of sendmsg):
int sendmsg(int fd, struct msghdr * msg, int flags);
struct msghdr {
void * msg_name; /* optional address */
int msg_namelen; /* size of address */
struct iovec * msg_iov; /* scatter/gather array */
int msg_iovlen; /* # elements in msg_iov */
void * msg_control; /* ancillary data, see below */
int msg_controllen; /* ancillary data buffer len */
int msg_flags; /* flags on received message */
};
struct iovec {
char* iov_base; /* The base address of the fragment */
int iov_len; /* Length in bytes of the fragment */
};
int foo(int fd, struct iovec *array, int array_len) {
struct msghdr msg = { 0, 0, array, array_len, 0, 0, 0};
// Make the msg_iov be FSEQ
struct iovec * foo = msg.msg_iov + 1;
return sendmsg(fd, &msg, 0);
}
#pragma ccuredwrapper("sendmsg_wrapper1", of("sendmsg"))
__inline static
int sendmsg_wrapper1(int fd, struct msghdr * msg, int flags) {
return sendmsg(fd, __ptrof(msg), flags);
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
We still get the error:
ccuredcode.tmp/ex44.c:1: Error: sendmsg appears to be external
(it has a wrapper), yet it has a mangled name: sendmsg_scsfs_.
Did you forget to use __ptrof and a version of __mkptr?
For more information, consult the online documentation on
"Writing Wrappers".
This happens because the wrapper and the true sendmsg still share all of
the pointer nodes in the struct msghdr and struct iovec. We must have
two versions of at least struct msghdr: one that the client code uses and
one that the library is using. If you browse the above code and look at the
prototype for the sendmsg function (not the wrapper) you will see that
CCured has tried to address this issue by creating the structures
msghdr_COMPAT and iovec_COMPAT (called _COMPAT because they
are reachable from the prototype of the wrapped function). But that was not
sufficient in this case: if you browse the code you will see that even the
msg_iov field of the new msghdr_COMPAT is FSEQ. And you get
the following error message:
Error: The suffix for the compatible version of msghdr_COMPAT is sfs.
This means that you have misused this compatible version. Please check your code.
The problem in this case is subtle: because sendmsg is declared to take a
struct msghdr argument and the result type of __ptrof is void *,
the CIL front-end has inserted a cast (struct msghdr*) in the call to
sendmsg in the wrapper. When CCured later decides to change the prototype
of sendmsg it does not change the cast also, which makes it appear that an
argument of type struct msghdr * is assigned to a formal of type struct
msghdr_COMPAT. To prevent this from happening, you must store the result of
the __ptrof function in a local variable, whose type you write:
struct msghdr __COMPAT *. Note that __COMPAT is here an attribute
to the structure name, not part of the name; CCured will do the right thing
when it creates the compatible version of msghdr. So, we try the
following:
int sendmsg(int fd, struct msghdr * msg, int flags);
struct msghdr {
void * msg_name; /* optional address */
int msg_namelen; /* size of address */
struct iovec * msg_iov; /* scatter/gather array */
int msg_iovlen; /* # elements in msg_iov */
void * msg_control; /* ancillary data, see below */
int msg_controllen; /* ancillary data buffer len */
int msg_flags; /* flags on received message */
};
struct iovec {
char* iov_base; /* The base address of the fragment */
int iov_len; /* Length in bytes of the fragment */
};
int foo(int fd, struct iovec *array, int array_len) {
struct msghdr msg = { 0, 0, array, array_len, 0, 0, 0};
// Make the msg_iov be FSEQ
struct iovec * foo = msg.msg_iov + 1;
return sendmsg(fd, &msg, 0);
}
#pragma ccuredwrapper("sendmsg_wrapper2", of("sendmsg"))
__inline static
int sendmsg_wrapper2(int fd, struct msghdr * msg, int flags) {
struct msghdr __COMPAT *msg1 = __ptrof(msg);
return sendmsg(fd, msg1, flags);
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
Using __ptrof is not enough, however, since the object still has
a FSEQ field. CCured detects this and changes the compat version
to have the same FSEQ field, so we get the same error as before.
What we should do instead is to make a copy of the structure pointed to
by msg and copy the fields one by one. There are several ways in which
CCured can help with that. First, there are some macros that you can use.
You can replace the declaration of msg1 above with:
__DECL_COMPAT_STACK(msg1, msghdr, msg);
This macro (defined and explained in ccured.h) first
reserves some storage for a struct msghdr_COMPAT on the stack (hence the
STACK in the name). Then it declares the msg1 variable and it copies the
contents of the struct msghdr pointed to by msg into the local copy.
This copying is done only if necessary (the msg pointer is not null and
the structure it points to is not already compatible). Here is the expansion
of the above macro invocation:
/* Declare the __deepcopy function that we need */
void __deepcopy_msghdr_to_compat(struct msghdr __COMPAT * compat, struct msghdr * fat);
/* Declare the place where we'll make the copy */
struct msghdr __COMPAT msg1_area;
struct msghdr * msg1__ptrof = __ptrof_nocheck(msg);
struct msghdr __COMPAT * msg1 =
/* We are done if we have NULL or an already compat struct */
(msg1__ptrof && (! CCURED_HAS_EMPTY_MANGLING(struct msghdr))) ?
/* Now do the copying as specified in the argument */
(__deepcopy_msghdr_to_compat(& msg1__area, msg1__ptrof), & msg1__area)
: /* No copying is needed. Use a trusted_cast to prevent CCured from
connecting the two versions of the structure */
(struct msghdr __COMPAT *)__trusted_cast(msg1__ptrof);
Notice that care is taken to avoid copying if not necessary and also to
prevent CCured from connecting directly the two versions of the struct.
There are more macros like __DECL_COMPAT_STACK, that allow you to
allocate the space on the heap instead of the stack, or even to avoid copying.
We'll discuss these later. Now we look again at the sendmsg wrapper in
which we use the new macro:
int sendmsg(int fd, struct msghdr * msg, int flags);
struct msghdr {
void * msg_name; /* optional address */
int msg_namelen; /* size of address */
struct iovec * msg_iov; /* scatter/gather array */
int msg_iovlen; /* # elements in msg_iov */
void * msg_control; /* ancillary data, see below */
int msg_controllen; /* ancillary data buffer len */
int msg_flags; /* flags on received message */
};
struct iovec {
char* iov_base; /* The base address of the fragment */
int iov_len; /* Length in bytes of the fragment */
};
int foo(int fd, struct iovec *array, int array_len) {
struct msghdr msg = { 0, 0, array, array_len, 0, 0, 0};
// Make the msg_iov be FSEQ
struct iovec * foo = msg.msg_iov + 1;
return sendmsg(fd, &msg, 0);
}
#pragma ccuredwrapper("sendmsg_wrapper3", of("sendmsg"))
__inline static
int sendmsg_wrapper3(int fd, struct msghdr * msg, int flags) {
__DECL_COMPAT_STACK(msg1, msghdr, msg);
return sendmsg(fd, msg1, flags);
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
If you browse this code you will see that in addition to the macro expansion
there is now a definition for the function
__deepcopy_msghdr_to_compat. CCured recognizes this name in a
prototype (which is added by the __DECL_COMPAT macros) and fills in the
body with code that tries to copy one field at a time from a struct msghdr
to a struct msghdr_COMPAT.
Here is how CCured fills in the code of deepcopy functions:
-
For any fields (except those of array types) that do not contain
pointers (i.e. a scalar field, or a substructure with no pointers), an
assignment is generated.
- For a pointer field, CCured generates code that detects whether a
deepcopy is necessary (the pointer is non-null and it points to a structure
that is not compatible). If a deepcopy is not necessary then an assignment is
generated (with care to avoid connections between the compatible and
incompatible structures). If a deepcopy would be necessary, CCured generates
code that aborts the execution with an error message.
- For an array field, CCured generates a loop that iterates over the
elements and generates code that copies each element.
- For a structure field, CCured first checks if a corresponding deepcopy
function has been declared or defined. If so, it produces a call to that
function. Otherwise it iterates over the fields and produces code to copy the
fields.
- For other kinds of pointers, CCured fails at compile time.
For our example, the deepcopy code that CCured generates would do. But such
code might not be what you need. If the iov_base pointer in the iovec
struct becomes non-SAFE then the default deepcopy function will abort. In
that case, you can write the parts of the deepcopy function that you care
about and let CCured fill in the rest. In the next example we show the full
wrapper for sendmsg, that works even when the embedded struct iovec is
mangled:
int sendmsg(int fd, struct msghdr * msg, int flags);
struct msghdr {
void * msg_name; /* optional address */
int msg_namelen; /* size of address */
struct iovec * msg_iov; /* scatter/gather array */
int msg_iovlen; /* # elements in msg_iov */
void * msg_control; /* ancillary data, see below */
int msg_controllen; /* ancillary data buffer len */
int msg_flags; /* flags on received message */
};
struct iovec {
char* iov_base; /* The base address of the fragment */
int iov_len; /* Length in bytes of the fragment */
};
int foo(int fd, struct iovec *array, int array_len) {
struct msghdr msg = { 0, 0, array, array_len, 0, 0, 0};
// Make the msg_iov be FSEQ
struct iovec * foo = msg.msg_iov + 1;
return sendmsg(fd, &msg, 0);
}
extern void* malloc(int);
extern void free(void*);
#pragma ccuredwrapper("sendmsg_wrapper4", of("sendmsg"))
__inline static
int sendmsg_wrapper4(int fd, struct msghdr * msg, int flags) {
__DECL_COMPAT_STACK(msg1, msghdr, msg);
int result = sendmsg(fd, msg1, flags);
// We can now free the msg_iov (if it was allocated)
if(msg1->msg_iov != msg->msg_iov) {
free(msg1->msg_iov);
}
return result;
}
__inline static
__DEEPCOPY_TO_COMPAT_PROTO(iovec) {
compat->iov_base = __ptrof_nocheck(fat->iov_base);
}
__inline static
__DEEPCOPY_TO_COMPAT_PROTO(msghdr) {
// We leave the msg_name and msg_control to CCured
if(CCURED_HAS_EMPTY_MANGLING(* fat->msg_iov)) {
// We do not need to copy the msg_iov array
compat->msg_iov = __ptrof(fat->msg_iov);
} else {
int len = fat->msg_iovlen;
int v;
compat->msg_iov = malloc(len * sizeof(compat->msg_iov[0]));
for (v=0; v<len; v++) {
struct iovec __COMPAT *iptr = __trusted_add(compat->msg_iov, v);
__deepcopy_iovec_to_compat(iptr, & fat->msg_iov[v]);
}
}
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
Notice several important things in the bodies for the deepcopy functions:
-
We use special macros to declare the prototype. These macros always
introduce two formals, one called “compat” (of type pointer to compatible
structure), and one called “fat” (of type pointer to regular structure). It
is a good idea to use these macros, because they ensure that at least the
prototype is right.
- In the TO_COMPAT function we always assign to the fields pointed to
by the compat formal.
- In each function we assign only the fields for which we want to override
CCured's automatically generated code. CCured scans the body that you wrote
and looks for assignments of the form compat->fieldname = .... For all
such fields that it finds, it assumes that you are taking care of that field
and does not generate code. Note that you are then responsible for writing the
code for all elements of that field (if it is a structure or array)! Note also
that you should not change the value of the compat formal and should not
use some other variable to do the assignment, or else CCured will not notice
it.
- As in any wrapper, you must be careful to not mix the two versions of
the structure. Thus you must use __trusted_cast and __ptrof to do
assignments between pointers to different versions of structures. Notice also
that we use __trusted_add instead of pointer arithmetic on the
compatible pointer types.
- You should avoid copying as much as possible (i.e. use the
CCURED_MANGLING_OF macro).
- The code that CCured generates automatically is inserted before
the code that you write. But you should not rely on that!
Now we look briefly at the companion method recvmsg. Instead of sending a
message, it receives one. This means that the msg_iov data is an output
value, not an input. We use the following wrapper in this case:
int recvmsg(int fd, struct msghdr * msg, int flags);
struct msghdr {
void * msg_name; /* optional address */
int msg_namelen; /* size of address */
struct iovec * msg_iov; /* scatter/gather array */
int msg_iovlen; /* # elements in msg_iov */
void * msg_control; /* ancillary data, see below */
int msg_controllen; /* ancillary data buffer len */
int msg_flags; /* flags on received message */
};
struct iovec {
char* iov_base; /* The base address of the fragment */
int iov_len; /* Length in bytes of the fragment */
};
int foo(int fd, struct iovec *array, int array_len) {
struct msghdr msg = { 0, 0, array, array_len, 0, 0, 0};
// Make the msg_iov be FSEQ
struct iovec * foo = msg.msg_iov + 1;
return recvmsg(fd, &msg, 0);
}
extern void* malloc(int);
extern void free(void*);
#pragma ccuredwrapper("recvmsg_wrapper", of("recvmsg"))
__inline static
int recvmsg_wrapper(int s, struct msghdr *fat_msg, int flags) {
__DECL_COMPAT_STACK(lean_msg, msghdr, fat_msg);
int result = recvmsg(s, lean_msg, flags);
// Copy the contents of lean_msg into fat_msg
__COPYOUT_FROM_COMPAT(lean_msg, msghdr, fat_msg);
if(lean_msg->msg_iov != fat_msg->msg_iov) {
// If we allocated new space for msg_iov, free it now
free(lean_msg->msg_iov);
}
return result;
}
__inline static
__DEEPCOPY_FROM_COMPAT_PROTO(iovec) {
// Notice how we use iov_len to construct the metadata for the iov field
fat->iov_base = __mkptr_size(compat->iov_base, compat->iov_len);
}
__inline static
__DEEPCOPY_FROM_COMPAT_PROTO(msghdr) {
fat->msg_name = __mkptr_size(compat->msg_name, compat->msg_namelen);
fat->msg_control = __mkptr_size(compat->msg_control, compat->msg_controllen);
if(CCURED_HAS_EMPTY_MANGLING(* fat->msg_iov)) {
// We do not need to copy the msg_iov array
fat->msg_iov = __mkptr_size(compat->msg_iov,
compat->msg_iovlen * sizeof(fat->msg_iov[0]));
} else {
int len = compat->msg_iovlen;
int v;
fat->msg_iov = malloc(len * sizeof(fat->msg_iov[0]));
for (v=0; v<len; v++) {
struct iovec __COMPAT *iptr = __trusted_add(compat->msg_iov, v);
__deepcopy_iovec_from_compat(& fat->msg_iov[v], iptr);
}
}
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
For recvmsg we use the companion functions that copy _FROM_COMPAT.
We declare the deepcopy functions as before, except that now we must assign to
the fat fields, and we must use the __mkptr family of functions to
construct the metadata. Notice that for the output argument of recvmsg we
copy the data from the stack-allocated lean_msg into msg.
The above macros are written without close attention to memory leaks.
However, these are just examples that are intended to show you how to write
wrappers.
The CCured library defines the functions
__deepcopy_stringarray_from_compat and
__deepcopy_stringarray_to_compat that operate on nul-terminated
sequences of pointers to nul-terminated strings (such as argv in the
prototype for main). To use these functions you must add:
#include "functions/deepcopy_stringarray.h"
8.4 Wrappers for data
So far we looked at wrappers for functions. Sometimes the global that needs
to be wrapper is not a function but a name for a data area. In the standard
library that is very rare. One example is the environ global variable of
type char ** that stores a pointer to a null-terminated sequence of
environment strings (like the sometimes-used third argument of main).
If you access this variable directly you might get an error like this:
tcpserver_comb.o: In function `env_get_qf':
tcpserver_comb.o(.text+0x15d10): undefined reference to `environ_qq'
You cannot write wrappers for data! You MUST replace the access to the data
variable with a function. There are many ways in which you can write the
function. If the wrapped global had type char * you could just write a
function:
// This is for the cast when environ is a pointer to a string
char* get_environ(void) {
return __mkptr_string(environ);
}
But this would not be enough for our case. In general, both of the pointer
types in environ could be fat, so you must do a deepcopy:
#include "functions/deepcopy_stringarray.h"
// This is Ok, but allocated too often
char** get_environ(void) {
return __deepcopy_stringarray_from_compat(environ);
}
If your program accesses the environ variable a lot, you do not want to
make a deepcopy every time. So, you can make one and save it in a variable,
which you can then access at will.
Or, maybe your program accesses environ always with an index operation
environ[i]. In that case you can avoid deepcopying altogether by writing a
function environ_idx that takes an integer and returns the ith
element in environ. I show below the case where it appropriate to trust
that the index is within the bounds:
extern char ** environ;
char* environ_idx(int i) {
char * __SAFE * __SAFE p_environ = __trusted_add(environ, i);
// We are going to believe that i is within bounds
return __mkptr_string(* p_environ);
}
8.5 Wrapper Helper Functions
8.5.1 Helpers for incoming wrapper arguments
These functions have two purposes. First, they convert the incoming argument
(which may be a fat pointer) to a regular C pointer or data to be used in
run-time checks and for passing to C library functions. Second, these
functions direct CCured to request some minimal metadata to accompany the
pointer. For example, on entry to a string library function CCured will want
to request that the pointer be accompanied by extent, and the target of the
pointer be a null-terminated string buffer.
-
__ptrof_nocheck: Returns the pointer value from a fat pointer. No
constraints for type inference.  No
constraints for type inference. Check that the pointer is in bounds, if the
incoming pointer has bounds.
- __strlen: Returns the length of the string, not counting the
terminating nul. Calls ccured_fail if ptr is null, or out of bounds.
Requires that the argument be a string or a null-terminated sequence
pointer.
- __verify_nul: This is like __strlen, but ignores the
resulting length.
- __stringof: This is like __verify_nul followed by
__ptrof.
8.5.2 Helpers for outgoing wrapper return values
These functions allow you to construct fat pointers from regular C pointers.
These functions do not do any checking.
-
__mkptr_size.  Takes an argument that is assumed to be a
null-terminated string, and creates a fat pointer if needed.
8.6 Final Notes on Wrappers
-
The documentation for the wrapper helper functions is in
ccured.h (which lives in the cil/include directory). That file
is included automatically in all cured files.
- Wrappers are automatically polymorphic.
- Wrapper-replacement is not performed inside a wrapper body!
- At the moment, wrappers do not work well for WILD pointers. You can
still write those wrapper manually but the __mkptr functions do not work
well for those.
- At the moment, wrappers do not work well for variable argument
functions. However, this is possible in certain cases (see
stdio_wrappers.h for the printf wrapper).
- Note that there is no actual requirement that a wrapper call the
external function that it is intended to replace. If you wish, the wrapper can
call a different external function, or simply reimplement the entire routine.
For example, the wrapper for strdup that comes with CCured calls
malloc and strcpy rather than the Standard Library's strdup
because we wanted to ensure that any strings created could be managed by our
garbage collector.
- If you write wrappers for generally useful libraries (such as the
standard library), please give them to us so we can distribute them to other
interested persons.
- Remember that all of the structure types that are reachable from the
prototype of the wrapped function are converted to a compatible version, whose
pointers should be all of SAFE kind and whose name is obtained by adding to
the structure name the _COMPAT attribute.
- The __DECL_COMPAT macros all work as declarations, not
statements.
- The wrappers for sendmsg and recvmsg are in the file
include/socket_wrappers.h. Take a look also at
include/netdb_wrappers.h.
- When you write wrappers, do not use functions that you are not sure have
a prototype (this is why we put the wrappers at the end of the include files).
Otherwise, you will generate warning and unnecessary WILDness in the
code. If you need to use malloc, use wrapperAlloc instead (for
which we guarantee the existence of the prototype).
Chapter 9 Advanced CCured Issues
In this chapter we discuss a collection of issues having to do with using
CCured on real programs. Some of the issues are related to sound handling of
dark corners of the C programming language (e.g. function pointers,
initialization of globals, variable argument functions). Other issues are
related to mechanisms to give hints to the CCured inferencer with the ultimate
goal of reducing the number of cases in which the inferencer gives up and
decided to use the conservative but expensive WILD pointers (e.g.
polymorphism, custom memory allocators).
9.1 Function Pointers
One of the signs that a C program is a “serious” one is the use of function
pointers. There would be nothing wrong or unsafe about that if it wasn't also
the case that most programmers do not feel necessary to use accurate types for
function pointers, or to even use function prototypes. This is probably due to
the fact that the syntax for function types in C is terrible. How often have
you declared your function pointers to have type void (*)() when you
actually wanted to say int * (* (* x3))(int x)(float) (a pointer to a
function that takes an int and returns a pointer to a function that takes
a float and returns a pointer to an int).
Of course, misusing function pointers can lead to the worst kind of errors.
Fortunately such error rarely go unnoticed in code that is executed.
CCured supports two kinds of function pointers. The SAFE function pointers
can only be invoked with the same number of arguments. If the types of the
arguments are not right it is the argument that becomes WILD not the
function pointer. A SAFE function pointer can only be cast to an integer or
to the same function pointer type. We also have WILD function pointers
which you can (try to) use as you please. In fact a WILD function pointer
can be cast to any other WILD pointer type and can be stored in any tagged
area. For this reason its representation must match that of any WILD
pointer. However the capabilities of a WILD function pointer are typically
quite different from those of a regular function pointer. For example, you
should not be able to read or write from a function pointer.
The next picture shows the meaning of the _b field for a WILD
function pointer.
Any function whose address is taken and becomes WILD, or that is used
without a prototype (see the discussion at the end of this section) is a tagged function and has an associated descriptor that encodes the actual code
to the function and the number of arguments. Here is an example:
int taggedfun(int anint, int * aptr) {
return anint + * aptr;
}
int main() {
int * i = taggedfun; // Bad cast. wildfun becomes tagged
// Now we invoke it
((void (*)(int,int*))i)(5, i);
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
The structure of a function descriptor is as shown below and a pointer to the
_pfun field is used as the _b field whenever the address of the
function is taken.
struct __functionDescriptor {
unsigned int _len ; // Always 0
void (* _pfun)() ; // Pointer to a function
unsigned int _nrargs ; // The number of arguments
};
Since the _len field is always initialized to zero, whenever this
WILD pointer is used for a read or a write it would appear that it points
into a zero-length tagged memory area, so the bounds check will fail. We then
have to protect against the pointer being subject to arithmetic prior to
invocation. We do this by storing in the function descriptor the actual
pointer to the function and checking at the time of a call through a WILD
function pointer that the _p field of the pointer is equal to the
_pfun field in the descriptor.
Finally we have to ensure that the function is called with the right number
and kinds of arguments. There is no hope to be able to ensure this statically
because a WILD function pointer can be used very liberally as any other
WILD pointer. So, CCured conservatively forces
all arguments and the return type to be WILD pointers. This includes
arguments and return types that are actually scalars (see the example above
for how integers are wrapped into WILD pointers). This will ensure that the
types are the same (or compatible) and all we have to check is the right
number of arguments is passed to the function. To perform these checks we use
the following run-time support function:
/* Check that a function pointer points to a valid tagged function and check
that we are passing enough arguments. We allow the passing of more
arguments than the function actually expects */
__CHECK_FUNCTIONPOINTER(void *_p, /* The _p field of the function pointer */
void *_b, /* The _b field */
int nrActualArgs); /* The number of actual arguments */
Also, always use prototypes for the external functions you are using.
Otherwise, it will appear to CCured that you are casting the function pointer
to various incompatible types corresponding to each use and the function will
be declared tagged (and pointers to such function to be WILD). You get some
help from CCured here because its whole-program merger will construct
prototypes for the functions that are defined somewhere in your program. But
when you use even simple things like printf you must include the proper
header files.
9.2 The main Function
The main function is the entry point to your program. The most general
type of the function is:
int main(int argc, char **argv, char **envp);
although when the arguments are not used it is common to not write them.
Depending on how you use the argv and envp arguments, CCured might
decide that they should be of some non-SAFE type. In that case CCured will
generate code that makes copies of appropriate kind of the argc and
envp arguments.
Take a look at what happens for this example:
int main(int argc, char **argv) {
for(; *argv; argv ++) { // Scan the args
char *p = *argv;
while( *p) { p ++; }
}
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
CCured will also insert a call to ccuredInit, which initializes the
CCured run-time library.
9.3 Global Initialization
C has a very rich language of initializers for globals and locals. The
language is so rich that neither gcc nor MSVC implement it fully. For a
discussion of how our front-end handles initialization, please see the the CIL documentation.
Once programs are presented to CCured all the initialization for locals is
turned into assignments, but most initialization code for globals is
preserved. However, in some cases CCured must insert some checks related to
the initializers. These checks are placed in a special function called a global initializer.
The name of a global initializer starts with __globinit. CCured will
try to insert a call to the global initializer that it creates in the main
function to ensure that it is run before anything else in the program. If it
cannot find a main it will emit a warning:
Warning: Cannot find main to add global initializer __globinit_myfile
If you see such warnings and intend to actually run the code, make sure
whoever invokes any function in the cured code calls the global initializer
first.
9.4 Casting Integers to Pointers
In CCured it is Ok to cast any kind of pointer to an integer, and in fact any
pointer comparison is performed after such a cast. But if you try to cast an
integer to a pointer the following two things happen:
-
The inferencer will not allow the destination pointer to be SAFE
unless the value you are casting is the constant 0. Any other kind of pointer
can be used.
- More importantly, the resulting pointer will have metadata that will
prevent you from using the pointer in a memory dereference. For a FSEQ
pointer the _e field will be null, for a SEQ both the _b and the
_e fields will be null and for a WILD the _b field will be null.
This means that such pointers cannot be used in memory dereferences. If your
program casts a pointer into an integer and then back to a pointer this will
be an issue. CCured will emit a warning whenever this happens. So far we have
very few programs that do this and even then in one of few forms.
Some programs are just not careful about keeping pointers separate from
integers and gratuitously cast to integers. The solution in that case is to
change the type of the intermediate location to a void* (or to a more
precise type of pointer if possible).
Other programs cast pointers to integers because they want to do pointer
arithmetic and do not have to worry about the implicit scaling that C uses for
pointer arithmetic. Use char* to do such arithmetic.
Some other programs also want to do arithmetic but of a kind not allowed for
char* such as the following code which tries to align a pointer to a 16
byte boundary:
int* alignit(int *x ) {
return (int*)((int)x & ~15);
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
The solution here is a cute trick that can be used in some situations to cast
an integer into a pointer, provided you know that it has the same
metadata with some other legal pointer. Thus to cast the integer x to a
pointer, while borrowing the metadata from pointer pdo:
p + (x - (int)p);
Thus you are turning a cast into pointer arithmetic. CCured will force the
kind of the pointer to be either WILD or SEQ but everything will work as
expected. Of course you have to worry about scaling back the difference by the
size of the type pointed to by x. Here is how the previous alignit
function can be written:
int* alignit(int *x ) {
int ix = (int)x & ~15;
return x + ((ix - (int)x) >> 2);
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
If you are lazy and do not want to change your code you can ask CCured to
insert code that at every cast from a scalar to a pointer records the
line-number of the cast. Then when a non-pointer is dereference the CCured
run-time system will try to tell you which particular cast in your program
produced this fake pointer. Use the interceptCasts pragma for this purpose
(see Section 9.10). We have not found this feature very useful because in
fact not too many programs cast integers to pointers.
9.5 The sizeof Issue
It is obvious by now that CCured will change the layout of some datatypes.
That can lead to several kinds of problems. For example, if you are calling a
library that is not cured then you better not change the layout of the data
that is passed back and forth. This issue is discussed more in
Chapter 8. Another problem is when the code is written assuming that
datatypes have a certain layout, such as the following code that accesses the
#include <stdio.h>
int *a[8];
void bad_code(int * *x) {
int * pa = a; // Make a's elements WILD
printf("a has %d elements", sizeof(a) / 4);
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
This code will probably print 16 instead of 8 because each element of a
is now 8 bytes long. Such code is very ill and it cannot be cured without
manual intervention. So, let's assume that we change the code to:
#include <stdio.h>
int *a[8];
void so_and_so_code(int * *x) {
int * pa = a; // Make a's elements WILD
printf("a has %d elements", sizeof(a) / sizeof(int *));
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
As it turns out this code is perfectly fine but our inferencer cannot tell
that there is a connection between the type of the element of a and the
int * that appears in the argument of sizeof. Even though the bad cast
will force the array elements to be WILD pointers the int * that
appears in the argument of sizeof will be a SAFE pointer. Thus this
code will also print 16. In fact, you will see a warning:
pathexec_env.c:42: Warning: Encountered sizeof(int */* __attribute__((___ptrnode__(2595))) */) when type contains pointers. Use sizeof expression. Type has a disconnected node.
If CCured says that the type has a connected node, then you are probably Ok.
It means that the node inside sizeof is connected to the other nodes, so it
will probably get the right kind. However, if CCured says that the type has
disconnected nodes then you should worry.
To really point out the connection change the code as shown below. It can be
argued also that this code is clearer and thus should be used even if you do
not use CCured.
#include <stdio.h>
int *a[8];
void good_code(int * *x) {
int * pa = a; // Make a's elements WILD
printf("a has %d elements", sizeof(a) / sizeof(a[0]));
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
Similar problems arise for any use of sizeof, such as in the argument of
allocation functions.
9.6 Variable argument functions
Variable-argument functions in C are inherently unsafe since there is no
language-level mechanism to ensure that the actual arguments agree in type and
number with the arguments that the function will be using. There are several
ways to implement variable argument functions in C and CCured supports some of
them quite well:
-
You can try to implement such function by taking the address of a formal
argument and working your way through the stack frame. If you do this we wish
you luck and perseverance and you'll need them both while trying to figure out
why your program does not behave for compilers such as gcc. CCured
certainly cannot allow this because this is exactly the kind of operation that
viruses would want to perform.
- A better alternative is to use the __builtin_next_arg function on
gcc and then work your way up the stack. CCured does not support this
low-level implementation of variable argument functions, but see next.
- You can write your variable argument functions using the standard macros
in <stdarg.h> and <vararg.h>. CCured supports most such functions with
minimal programmer intervention.
There are two kinds of variable-argument functions in C:
-
Those that take an arbitrary number of arguments following the last
specified formal (their function type contains ... after the last formal).
We'll call these functions vararg functions. An example is printf:
int printf(const char* format, ...)
- Those that take as arguments one or more pointers to lists of arguments.
We'll call these functions valist functions. An example is vprintf:
int vprintf(const char* format, va_list args)
CCured supports both kinds of functions and will scan the program to find out
for each function what types of arguments are passed. In Section 9.6.1
we describe how the programmer can prevent this automatic inference by
specifying the set of types of arguments.
CCured redefines the macros in <stdarg.h> and <vararg.h> to do
special bookkeeping. In vararg functions, the macro va_start is used to
initialize an va_list variable to point to the trailing arguments. CCured
checks that the second argument is the last formal before the ....
Both in vararg and valist functions the macro va_arg can be used, as
follows:
T x = va_arg(args, T)
args must be a va_list variable and T must be compatible after
the usual actual argument promotions (e.g. char and short to int
and float to double) with one of the types in the struct associated
with args. CCured checks this at run-time.
The CCured support for variable argument functions is quite flexible.
Multiple variable argument lists can be processed in parallel, an argument
list can be re-initialized with va_start and processed multiple times. A
function can even work with variable argument lists that have different sets
of types accepted (but for this you need to specify manually the set of types
of arguments as explained in Section 9.6.1). Variable argument lists
can be passed down but the regular CCured checks for stack allocated variables
will prevent the passing of these lists up the call chain and also their
storing in the heap.
The main thing that is not supported in CCured is the fetching of an argument
with a different type than it was stored. It remains to be seen if this is a
problem. We have looked at several variable argument functions (including full
implementations of printf and sprintf) and so far we have found that
CCured accepts those functions without any change except for the specification
of the struct of the accepted argument types (as explained below).
9.6.1 Programmer control over vararg functions
If you do not want CCured to find automatically all the types that can be
passed to a function, you can specify the set of types that can be used for
arguments. Also, you should not let CCured infer the argument types for
printf-like functions, but you should instead use the special support for
them, as explained in Section 9.6.2.
You can declare the argument types by declaring a descriptor. This is a
struct data type whose fields have the types that can be passed to the
function. The order and the names of the fields do not matter. For example,
such a struct for printf would be the following (this structure is defined
in ccured.h):
struct printf_arguments {
int f_int;
double f_double;
char *f_string;
};
The simplest way to specify that such a struct describes the types of
arguments for a variable argument function is to use a pragma:
#pragma ccuredvararg("myvarargfunction", sizeof(struct printf_arguments))
Notes:
-
Since most variable argument functions are printf-like, CCured
provides special support for them. See Section 9.6.2 if your function is
of this kind.
- Note that CCured predefines the struct printf_arguments so you do
not have to redefine it.
- Like all CCured pragmas, you can put them anywhere in your project since
CCured is going to merge all files into one. However, I typically prefer to
put such pragmas in header files near the prototype for the function.
- This method does not work for specifying pointers to variable argument
functions. You must use the attribute method described below.
- The sizeof operator is there because the syntax of pragmas is pretty
much that of function calls and thus we cannot use types directly.
- This pragma can be used with both kinds of
variable argument functions. In the case of valist functions it specifies that
all formals and locals of type va_list can hold arguments of the
given types.
An equivalent method is to associate the __CCUREDVARARG(struct
printf_arguments) attribute with the type of the function
myvarargfunction:
int (__CCUREDVARARG(struct printf_arguments) myvarargfunction)(int last, ...);
You have to use this method if you want to specify that a function pointer is
variable argument:
int (__CCUREDVARARG(struct printf_arguments) * myvarargptr)(int last, ...);
typedef int (_CCUREDVARARG(struct printf_arguments) fptr)(char *format,...);
A more fine-grained way to specify the same thing is to use the
__CCUREDVARARG type attributes for va_list every time it appears.
This allows you to specify different sets of types for different locals:
va_list __CCUREDVARARG(struct printf_arguments) args1,
__CCUREDVARARG(struct some_other_type) args2;
9.6.2 Printf-like functions
Since the vast majority of uses of variable argument functions if for
printf-like functions, CCured contains special support for them.
Specifically if a vararg function is declared to be a printf-like function
then all of its invocations in which the format string is a constant will be
checked statically. For the other invocations a wrapper for printf will be
called that will check the types of the actuals against the format string
before calling the real printf function.
To declare a function to be printf-like use the following pragma:
#pragma ccuredvararg("myprintf", printf(1))
where the last argument is the index of the format argument in the argument
list (starting with 1). Note that you will get a run-time error if you try to
use the va_arg macro in the implementation of such a function. In those
implementations you should invoke functions like vprintf and vsprintf
instead.
GCC already has support for communicating to the compiler that a function is
printf-like. This is done as follows:
int myprintf(const char* format, ...) __attribute__((format(printf, 1, 2)))
where the “1” means that the first argument is the format string and the
“2” means that we should start checking with the second argument. CCured
recognizes this attribute and it considers it equivalent with the
ccuredvararg from above. Note that the second argument in the
format attribute is ignored in CCured.
You can use the format attribute even for function pointers:
int (__attribute__((format(printf, 1, 2))) *myptr)(char *format, ...);
Note that CCured does not currently like passing pointers to printf with
the intention of printing the pointer value. You should manually cast those
pointers to long when passing them to printf-like functions.
Also, you should not let CCured infer automatically the descriptors for
printf-like functions. Otherwise, it is quite likely that the descriptor
that will be inferred is different than the built-in descriptor
printf_arguments (which the runtime library is using to check the calls
to printf-like functions. CCured will warn you about all automatically
inferred descriptors and you should manually inspect all the functions
involved.
As for the regular variable argument functions, the pragma works only for
named functions but not for pointers to functions. For that purpose you must
use attributes:
int (__CCUREDFORMAT(1) * myprintf)(char *format, ...);
typedef int (_CCUREDFORMAT(1) fptr)(char *format,...);
9.6.3 Scanf-like functions
Since it proved too much trouble to handle scanf-like functions in a safe
yet transparent way we currently require the programmer to rewrite the
invocations to scanf using a number of functions that we provide. For
example instead of
int entry; double then; char buffer[6];
... fscanf(file, "Entry:%d; Then:%lf; 5 digits:%5[0-9]; useless text.",
&entry, &then, buffer) ...
you should write
... (resetScanfCount(),
entry = ccured_fscanf_int(file, "Entry:%d"),
then = ccured_fscanf_double(file, "; Then:%lf"),
ccured_fscanf_string(file, "; 5 digits:%5[0-9]", buffer),
ccured_fscanf_nothing(file, "; useless text."), //advance the file pointer.
getScanfCount ()) ...
The functions resetScanfCount and getScanfCount are necessary only
if you use the result of the call to fscanf in the original code. Note
that our replacement scanf functions can be used to return only one result
at a time, consequently the format string that is passed must contain only one
format specifier, possibly along with characters to be matched.
The following are the scanf-like functions that we currently support:
extern int ccured_fscanf_int(FILE *, char *format);
extern double ccured_fscanf_double(FILE *, char *format);
extern void ccured_fscanf_string(FILE *, char *format, char *string);
extern void ccured_fscanf_nothing(FILE *, char *format);
If the original program uses scanf, just consider that you are using
fscanf from stdin. If instead your program contains sscanf then
you can use the function
void resetSScanfCount(char *string);
to dump the string to the temporary file ccured_sscanf_file
then use the replacement for fscanf from above. For example,
... (resetSScanfCount(inputString),
entry = ccured_fscanf_int(ccured_sscanf_file, "Entry:%d"),
then = ccured_fscanf_double(ccured_sscanf_file, "; Then:%lf"),
ccured_fscanf_string(ccured_sscanf_file, "; 5 digits:%5[0-9]", buffer),
getScanfCount ()) ... //getScanfCount is required when using resetSScanfCount
Note that the current support for scanf is far from satisfactory and
will likely change in the future
9.6.4 Implementation Issues
Almost all of the checking for variable-argument functions is done at
run-time. At the time of a call each actual argument is compared with the
types in the struct associated with the vararg function. A global data
structure is filled with the number of arguments (in the global
__ccured_va_count and a list of indices describing for each actual
argument the index within the struct types (in __ccured_va_tags).
In the body of a vararg function, a data structure is allocated on the stack
to hold a copy of the global description of the arguments that was created by
the caller. The call to va_start initializes this data structure and each
call to va_arg checks that we are not reading past the end of the
actuals and also that the type of the fetched argument matches that of the
actual argument.
9.7 Tagged Unions
As we have seen in Section 3.2.2 CCured can handle union types whose
fields have compatible pointer types at corresponding offsets. If this
is not the case then you will need to tell CCured how to handle the
union. One option is to turn the union into a struct, but we do not
recommend this because it increases memory usage and can change the
behavior of your program if your code writes to one union field and
then reads from a different one. A better option is to declare that
the union is a tagged union. CCured actually supports two
forms of tagged unions: one in which CCured adds a tag field and
maintains it for you, and one in which your program maintains its own
tag, and CCured checks that it is used properly.
9.7.1 CCured-maintained tags
You can declare a union to be tagged by adding the attribute
__TAGGED to its definition. CCured will expand the union to contain
a tag field. A tag is an RTTI value (Section 7.3) that encodes the
type of the last field written in each union value. Here is an example:
union int_or_ptr {
int i;
int *p;
} __TAGGED; // We declare it tagged
int main() {
union int_or_ptr x;
int i;
x.i = 5;
i = x.i; // This will work
i = * x.p; // This will fail
x.p = &i;
i = x.i; // This will fail
i = * x.p; // This will work
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
You can see that CCured has defined the following structure:
struct tagged_int_or_ptr {
struct RTTI_ELEMENT * __tag ;
union int_or_ptr __data ;
} __TAGGED ;
You can also see in the code that CCured generates assignments to the
__tag field before each assignment to a union field. And CCured inserts
calls to CHECK_UNIONTAG before each read-access to the field.
Notes:
-
If you declare a union type to be tagged you cannot take the address of
one of its fields. If your program takes the address of a union
field, you should change the union to a struct or otherwise
restructure the code.
- If a field of a union type is a structured type itself, any access to an
element of the field is treated (from the point of view of the tag
manipulation) as an access to that entire field. This means that you cannot
take the address of an element of a field either.
- You can call the function CCURED_HASUNIONTAG to test the
runtime value of a union tag. “CCURED_HASUNIONTAG(foo.bar)”
will return true if it is safe to read field bar in object
foo, or false if such a read would cause a run-time error.
- Tagged unions use the same tag mechanism as RTTI pointers
(Section 7.3). This allows interoperability with RTTI pointers,
and creates an exception to the general rule that you may only read
from the most recently-written field of a tagged union. If two
fields have pointer types, and one of those types is a subtype of
the other, it is possible to write to one field and then read from a
different field. (All pointers are subtypes of void*. Other
subtyping relationships can be declared as described in
Section 7.3.) For example:
typedef struct foo {
int *f1;
} Foo;
union ptrs {
void * v;
Foo * f;
int * i;
} __TAGGED; // We declare it tagged
int main() {
union ptrs u1, u2;
Foo f;
//We write to the "void*" field, and read from the "Foo*" field.
//This works because rtti_ptr has Run-Time Typing Information.
void* __RTTI rtti_ptr = &f;
u1.v = rtti_ptr;
Foo* pf = u1.f; // This will work
int* pi = u1.i; // This will fail
//We can also write to the Foo* field, and read from the void* field a
// pointer that has RTTI info.
u2.f = &f; //write to the "Foo*" field
rtti_ptr = u2.v; // and read from the "void*" field
pf = (Foo*)rtti_ptr; //This checked downcast will succeed.
return 0;
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
9.7.2 User-defined tags
Many programs define their own tagged unions, in which a struct
contains a tag field and a union holding one of several types of
data. In these cases, using a CCured-supplied tag is redundant. You
can annotate a union to tell CCured what meaning you assign to various
tag values, and CCured will then check that the tags are maintained
properly.
When a tag is modified, the “data” portion of the structure will be
zeroed. Therefore, when writing both the tag and data portions,
programs must always modify the tag first, followed by the data.
When a program reads or writes the data part of a tagged union, CCured will
read the tag and check that it is appropriate for the union field
being accessed.
Tags are defined by annotating each union field with
__SELECTEDWHEN(exp) where exp is a boolean expression.
exp may contain integer arithmetic and comparisons, and it can refer
to the runtime value of a field in an enclosing struct by specifying
the name of that field. For example:
enum tags {
TAG_ZERO = 0,
};
struct host {
short tag; // 0 for integer, 1 for structure, 10--12 for pointer to int
union bar {
int anint __SELECTEDWHEN(tag == TAG_ZERO);
struct str {
int * * ptrptr;
float f;
} structure __SELECTEDWHEN(tag == 1);
int * ptrint __SELECTEDWHEN(tag >= 10 && tag <= 12);
} data;
} g;
int x;
int main() {
g.tag = 12; //Select g.data.ptrint
g.data.ptrint = &x;
int* px = g.data.ptrint; //To check that it's okay to access g.data.ptrint,
//CCured checks "g.tag >= 10 && g.tag <= 12"
return 0;
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
In this case, the __SELECTEDWHEN attributes tell CCured that the
field data.anint is active when the tag field is 0, the field
data.structure is active when the tag field is 1, and the field
data.ptrint is active when the tag field is between 10 and
12.
Notes:
-
__SELECTEDWHEN annotations can reference more than one tag
field. The tags all must have integral types, and all must be
fields of the struct that contains the union in question.
- You must ensure that the __SELECTEDWHEN annotations are
disjoint. That is, no two fields should be “selected” at once.
CCured does not yet check this.
- The requirement that tags be written only after the data value
is in many ways too restrictive. We are currently exploring other
solutions. If this is too restrictive for your program and you do
not need to maintain binary compatibility with external code,
consider letting CCured add its own tag, as described in
Section 9.7.1. CCured-maintained tags do not place
any limitation on write order, since CCured will have exclusive control
over the tag field that it cares about.
- When writing the tag field, CCured will zero the data field only
if the new tag is different from the old.
9.8 Annotated Lengths
Programmers can annotate array pointers with length attributes.
CCured will then use the annotated length whenever it needs to do a
bounds check on that pointer, instead of transforming the pointer into
a fat pointer. This has two advantages:
-
CCured will not need to change the representation of this
pointer, which improves compatibility with precompiled code.
- Using a CCured fat pointer to store the length of a buffer may
be redundant if the program already maintains a similar length value
near the pointer.
Length annotations are allowed in two situations: struct fields may
have length annotations that depend on the values of other fields in
that struct, and function parameters may have lengths that depend on
other parameters in that function. (NB: but the annotations on
function parameters are not yet implemented. Coming soon ...)
Only pointer types may be annotated. The annotation
__SIZE(exp) on a field means that the associated pointer is
exp bytes long, where the expression exp can involve integer
constants, arithmetic, sizeof, and the names of other fields in
the same struct. So __SIZE(1 + foo) means that the specified
pointer has a length that's one greater than the runtime value of
field foo in the same object.
__COUNT(exp) means that the pointer is exp elements long.
So when annotating a pointer with type T*, the
annotation __COUNT(exp) is equivalent to __SIZE(exp *
sizeof(T)).
Any field that is referred to by a __SIZE or __COUNT
annotation is a metadata field. When a metadata field is
modified, any pointer fields that depend on it are set to NULL. Therefore, when writing both the metadata and pointer fields, programs
must always modify the metadata first, followed by the pointer.
When an annotated pointer field is read, CCured will read any
metadata fields as well, and associate that length with the pointer.
When a pointer field is written, CCured will check that the buffer's
length is less than or equal to the length specified by the current
value of the metadata fields.
extern void* malloc(int);
#pragma ccuredalloc("malloc", sizein(1), nozero)
struct bar {
int nrInts;
int *ints __COUNT(nrInts);
};
struct foo {
int sizeBars;
struct bar * bars __SIZE(sizeBars);
};
// Now the function that uses it
void init(struct foo* pFoo) {
int nrBars = 5;
pFoo->sizeBars = nrBars * sizeof(* pFoo->bars);
pFoo->bars = (struct bar*)malloc(pFoo->sizeBars);
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
In this code, we first overwrite the field pFoo->sizeBars, which
automatically sets the field pFoo->bars to NULL. The next step is
to write a new pointer to the pFoo->bars field. During this
write, CCured will check that the pointer being written (in this case,
the result of malloc) is at least “pFoo->sizeBars” bytes long.
9.9 Memory Management
The high order bit: we use the Boehm-Weiser garbage collector.
TODO : finish this section
9.10 CCured Pragmas
The following pragmas are recognized by CCured. Note that pragmas can only
appear in between global declarations. Some of them are discussed in more
detail in following sections:
-
#pragma box(off) - Turn off curing. The code that is not cured does
not contribute constraints to the type inferencer. However the code is changed
slightly during the program transformation phase to make sure that whenever it
refers to global variables that were cured it uses their _p field.
- #pragma box(on) - Turn curing back on
- #pragma nobox("myfunc") - Turn curing off for the function
myfunc
- #pragma boxtext(...) - CCured will turn this pragma into the ...
text in the cured file
- #pragma ccuredpoly("myfunc1", "myfunc2", "struct foo") - CCured will
treat the myfunc1 and myfunc2 functions and the foo structure
polymorphically. See Section 7.1.
- #pragma ccuredalloc("malloc", nozero, sizein(1)) - CCured will treat
malloc as an allocation function whose length is passed in the first
argument and which does not zero the allocated area. See Section 7.2.
- #pragma ccuredalloc("calloc", zero, sizemul(1,2)) - CCured will treat
calloc as an allocation function whose length is passed are the product of
the first two arguments and which does zero the allocated area. See
Section 7.2.
- #pragma ccuredvararg("myfunc", sizeof(struct myfunc_arguments)) -
Declares myfunc to be a variable argument function that can be passed a
variable number of arguments each having one of the types of the fields of
struct myfunc_arguments. See Section 9.6.
- #pragma ccuredvararg("myprintf", printf(2)) - Declares myprintf to
be a printf-like function whose format string is in the second argument.
See Section 9.6.
- #pragma ccuredwrapper("foo_wrapper", of("foo")) -
Declares foo_wrapper to be a wrapper for foo. See
Chapter 8. Implies #pragma cilnoremove and #pragma
ccuredpoly for foo_wrapper.
- #pragma cilnoremove("func1", "var2", "type foo", "struct bar") -
Instructs CIL to keep the declarations and definitions of the function
func1 and variable var2, the definition of type foo and of
structure bar.
Chapter 10 CCured Warnings and Errors
As you use CCured you might encounter various kinds of problems. Most of
these are due to a combination of aggressive coding practices and CCured being
less smart than the programmer. (Note: this section is continuously being
expanded; if you do not see the answer to your question, or if the answer is
not helpful, let us know).
-
You see this warning message in the merging stage:
combine20_2.c:11: Warning: uniqueVarNames: Changing the name of local tmp___0 in main to tmp___1
This is nothing to worry about. It means that the merged has discovered a
naming error in the merged file and has fixed it.
- Also during merging, you see:
/usr/include/sys/socket.h:189: Error: Incompatible declaration for accept (4). Previous was at rblsmtpd.c:103 (0) (different type constructors: void vs. int )
This means that the merger has detected two incompatible declarations or
definitions of the same external global. You must fix that either by changing
all definitions and declarations to have the same type. In fact, in the
sources for gcc we found a few bugs like this where one function was defined
with one type and was declared and used with another type in some other file.
In some cases however, there is nothing wrong with the program but it just
happens to redefine a function already defined in the library (as in the
example from which we got the error message). The program redefines accepts in
one file and uses the library's accept in another file. When you put the
things together it does not make sense anymore. You should change the name of
your global in that case.
- Also during merging, you may see:
/home/weimer/cil/include/netdb_wrappers.h:110: Warning: The name gethostent_wrapper is used for two distinct globals
TODO.
10.2 Inference
-
You see this warning in the Inference stage:
chkshsgr.c:8: Warning: Calling function getgroups without proper prototype: will be WILD.
getgroups has type void * __attribute__((___ptrnode__(12))) /* /* missing proto */ */()
You should always have prototypes for the functions that you use. In fact,
CCured will be happy if you include the prototype in at least one of your
modules!
- Or, you see this
chkshsgr.c:8: Warning: Calling function _exit with 1 arguments when expecting 0: will be WILD.
_exit has type void ()
Here what happens is that exit is declared like “void _exit();”. This
is legal in C, and it allows you to call _exit with whatever arguments
you choose. CCured will accept that but the cost will be that the function
becomes a WILD function with some significant run-time cost to ensure that
you call it correctly. Better, use a correct prototype!
- CCured cannot handle soundly inline assembly. You will see this if you
try to cure a program that contains inline assembly.
iopause.c:68: Error: You did not turn on the handling of inline assembly. Better hide this assembly somewhere else!
You can turn on the (unsound) handling of inline assembly by passing the
argument –allowInlineAssembly to CCured. But you should try instead to
hide that inline assembly from CCured. You can put it in a file that CCured
does not see, for example (pass the name of that function with the
–leavealone argument to CCured).
- Sometimes you see these warnings:
pathexec_env.c:42: Warning: Encountered sizeof(char */* __attribute__((___ptrnode__(2595))) */) when type contains pointers. Use sizeof expression. Type has a disconnected node.
Section 9.5 explains what the problem could be here.
- Sometimes you get this warning in the inference stage:
Warning: Generated automatic vararg descriptor for log_d: struct autoVarargDescr_log_d : char const */* __attribute__((___ptrnode__(922))) */,
uid_t
If this is a printf-like function you should declare it!
This means that the function log_d is a variable argument function that
was not declared using a ccuredvararg pragma. In absence of such a pragma,
CCured examines all the call sites of that function and collects the set of
types for the arguments. You must inspect the definition of the function
involved to ensure that CCured has inferred correctly. If you see that this is
not the case, or if the function is a printf-like function, the you
should provide an explicit descriptor, as explained in Section 9.6.
- You get an error “type mismatch between called and use of va_arg
function”. One possible problem is that you have a printf-like function that
CCured did not recognize and it inferred for it a wrong set of possible
argument types. For example,
#include <stdio.h>
#include <stdarg.h>
void myprintf(int level, const char *fmt, ...)
{
va_list ap;
char msgbuf[2048];
va_start(ap,fmt);
vsnprintf(msgbuf, sizeof(msgbuf), fmt, ap);
puts(msgbuf);
va_end(ap);
}
int main()
{
int i;
myprintf(0, "Hello, %s! 2+2=%d\n", "world", 4);
return 0;
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
Here CCured will infer that the possible argument types for myprintf
contain only int and char * (based on the call). Yet, the built-in
vsnprintf uses a different set of argument types. To fix this problem you
must add:
#pragma ccuredvararg("myprintf", printf(2))
- You might also see a warning such as the following:
/home/necula/ccured/include/netdb_wrappers.h:329: Warning: Solver: changing User Specified SAFE node 1371 (the local variable p_ith_alias) to WILD
This means that CCured must change a user-specified SAFE pointer into
WILD. Open the browser (see Section 5.1) and type in the node number
in question. Have you annotated that pointer to be SAFE? If yes, then
CCured thinks otherwise. If the message is from a wrapper (like the above
example) then most likely is a symptom of a badly written wrapper.
- Another possible warning that you might see is:
/usr/include/sys/socket.h:156: Warning: sendmsg appears to be external
(it has a wrapper), yet it has a mangled name: sendmsg_scsws_.
Did you forget to use __ptrof and a version of __mkptr?
For more information, consult the online documentation on
"Writing Wrappers".
In this case CCured mangles the name of the sendmsg function (see
Section 8.1). The type of this function is
struct msghdr {
void *msg_name ;
socklen_t msg_namelen ;
struct iovec *msg_iov ;
int msg_iovlen ;
void *msg_control ;
socklen_t msg_controllen ;
int msg_flags ;
};
int sendmsg(int s, const struct msghdr *fat_msg, int flags);
The particular mangling suffix (_scsws_) means that the msg_iov
pointer must be WILD. We can investigate why this happens also using the
browser. We press “Show types” and then click on msghdr. We see that
indeed the msg_iov field is WILD and when we click on the red pointer
type we find out that the __trusted_add_iov issue is the cause of this
problem.
main.c:278: Warning: Casting scalar (1) to non-WILD pointer in main!
This warning says that the given scalar expression is assigned to a pointer
lvalue. This assignment probably has the effect that the lvalue is not
SAFE. During the assignment, CCured will produce a fat pointer value whose
metadata is NULL, which means that it cannot be used to access memory. In this
particular case, this is Ok, since nobody in its right mind would plan to use
the number 1 as an address.
But sometimes you have a pointer value that is stored in an integer variable
and you want later to use it as a pointer. This is not safe in CCured, because
the compiler cannot be sure where this pointer originated. There are several
ways to solve this one. First, maybe the scalar variable that is being
assigned should be a pointer variable after all. If you cannot do that then
you must come up with some other pointer variable that is supposed to be in
the same home memory area as the scalar. Then you can change the code, as
follows:
int * home;
int scalar;
int * p = (int*)scalar;
// Change the last line:
int * p = home + (((int)scalar - (int)home) >> 2);
// Or, even
int * p = __mkptr(scalar, home);
Anyway, it is best if you keep pointer values into pointer variables at
all times.
Error: The suffix for the compatible version of sigaction_COMPAT is ws. This means that you have misused this compatible version. Please check your code.
The sigaction_COMPAT is a copy of struct sigaction that CCured
creates automatically for use in wrappers. However, that structure should
remain compatible with the external version. When you get this error it means
that you have written an incorrect wrapper and somehow the constraints from
your program flow into the compatible version of the structure. You should use
the browser to find out why this happens.
Warning: isImported for CHECK_FORMATARGS, which is not even declared
TODO.
dnssec.c:236: Warning: Casting SAFE void* to FSEQ. The area is 1 word.
TODO.
- During linking you get the error:
mathopd_comb.o: In function `log_request':
mathopd_comb.o(.text+0x189fd): undefined reference to `asctime_qs'
If the missing function is one with a mangling suffix (see Section 8.1)
then this means that CCured has changed the calling convention of the
functions involved (in this case asctime) and you need to provide a
wrapper to match the calling convention to the library version. See
Chapter 8 for details on how to do that.
If the missing function has a “_t” mangling, then it is a WILD
function. You have either used the function without a prototype, or with a
type that does not match the prototype, or you have cast its pointer to
something strange. See if you can fix the prototypes (and fill in the argument
types in the function types).
However, if the missing function does not have a mangled name then it means
that somehow CCured has lost an external global. This is probably a bug in
CIL, most likely in the module that removes unnecessary locals and prototypes.
Try disabling that module by passing the –keepunused flag to CCured.
- You get this error while trying to use the merger.
gcc -D_GNUCC -o tcpserver -s tcpserver_comb.o cdb.a dns.a time.a unix.a byte.a
cdb.a: could not read symbols: Archive has no index; run ranlib to add one
You look at cdb.a and it is not a object file at all but, starts like
this:
!<arch>
cdb.o/<PHONE_NUMBER> 1002 100<PHONE_NUMBER>9 `
#pragma merger(0, "./cdb.i", "")
# 1 "cdb.c"
# 1 "/home/necula/ccured/include/gcc_2.95.3/sys/types.h" 1
You look at cdb.o and, indeed, it is not an object file, but a copy of
the cdb.i file. This is normal. What happened is that for merging,
compilation means just preprocessing (this explains why cdb.o is as it
is). But the problem is that you have just invoked the ar program to build
a library out of object files that are actually text file. The solution is to
use instead of ar the command ccured –mode=AR. This should create the
cdb.a library as needed, but if you look at it, it is a C source file that
contains all the stuff from the archived files, with the necessary renaming to
avoid conflicts between static variables. You can find more information about
the merger at ../cil/merger.html.
- You get this error while trying to use the merger.
ranlib: cdb.a: File format not recognized
You look at cdb.a and find that it is a merged source file. You should
not use ranlib on such files.
- Or this link-time error:
client.c:2139: warning: second parameter of `va_start' not last named
argument
TODO.
10.5 Running the Cured Code
When you run the code you might get run-time errors. Make sure you read the
Chapter 4 on ways to control the handling of errors.
- At run-time you get the error:
Failure UBOUND at config.c:924: new_pool(): Ubound
Aborted
This says that the code triggered an upper bound check. Investigate the
problem first by looking at the line number. Also run the program with
gdb; it should stop right where the error arises.
There is one relatively common instance when you get a UBound error. Say that
the variable ceiling is intended to be the end of a memory area:
int * ceiling = area + area_length;
The area pointer will be inferred FSEQ but the ceiling pointer
will be inferred SAFE. This means that before the assignment, CCured will
insert an upper-bound check, which will fail.
There are several solutions:
-
Change the ceiling pointer to be a FSEQ pointer. This is always
Ok, or
- Change the ceiling pointer to be an integer. This is Ok if you
never plan to use the value it contains as a pointer (scan the comb.c
file to see all the uses of a variable), or
- Pass the–noUnrefPointerChecks to CCured to ask it to figure out
those pointer values that do not appear to be used, and to treat them as
integers.
Failure UBOUND at ping.c:1303: main(): Ubound
Also an upper bound. We look at the code and we find the following:
struct icmp {
int various;
char data[1];
};
char outpack[65536];
char foo() {
// Get the 8th data character
return ((struct icmp*)outpack)->data[8];
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
This looks good overall, but technically we are accessing the 8th character
in a 1 character array. Once you cast the outpack array to struct
icmp*, you loose access to some of the trailing characters. If you look at
the cured code you will see in fact the CCured uses a length of 1 in doing the
bounds check.
Instead you could change the code to express more directly what you mean:
struct icmp {
int various;
char data[1];
};
char outpack[65536];
char foo() {
// Get the 8th data character
return * (outpack + (int) &((struct icmp*)0)->data[8]);
}
Browse the CCured inferred pointer kinds,
or see the CCured output
for this code fragment
Now the access is in the outpack array and its length is used for bounds
checking.
- You get this error:
Failure STORE_SP at pathexec_env.c:47: pathexec_qq(): Storing stack address
This means that you are trying to store into the heap, a pointer to some
stack location. See Section 3.5 for how to fix this. In rare occasions
the pointer is from your argv or envp arguments. Use strdup in
that case.
Copyright (c) 2001-2002,
-
George C. Necula<EMAIL_ADDRESS>- Scott McPeak<EMAIL_ADDRESS>- Wes Weimer<EMAIL_ADDRESS>All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
3. The names of the contributors may not be used to endorse or promote
products derived from this software without specific prior written
permission.

If you find a bug in CCured, please send email to
George Necula.
Below are some of the changes in the CCured system. These are in addition to
changes made to the underlying CIL infrastructure
-
August 6, 2006: Released version 1.3.5
- March 31, 2006: Fix bugs with global initializers and
errors that were being ignored in the choptim optimizer.
- November 1, 2005: Improved the __SIZE and
__COUNT annotations. See Section 9.8
- September 26, 2005: Reenabled special handling of strings.
This may break some code, so use –noStrings to turn it off.
- June 14, 2005: Tagged unions now use the same tags as RTTI
pointers, making them more flexible. See Section 9.7.
- January 11, 2005: Released version 1.3.2
- January 11, 2003: Fixed bug in strcpy wrapper.
- September 14, 2004: Fixed performance bug on gcc 3.0 and later.
- August 7, 2004: Released version 1.3.1
- July 28, 2004: Ocaml version 3.08 is required.
- February 20, 2004: Released version 1.2.4
- February 15, 2004: Inlined some of the mkptr and ptrof
wrappers.
- February 14, 2004: New implementation for discriminated unions.
- January 26, 2004: Changed wrapper helpers such as
__ptrof so that the can no longer be used to silently cast from
one type to another.
- January 19, 2004: Changed the statistics to not include unused
pointer nodes.
- November 25, 2003: Released version 1.2.2
- October 13, 2003: Fixed the handling of polymorphic structures.
- September 17, 2003: Released version 1.2.1. This release includes
many fixes in the underlying CIL infrastructure.
- April 8, 2003: Started version numbering for CCured. Released
version 1.1.0, based on CIL 1.1.0.
- April 7, 2003: Fixes in the error messages and the control of
error logging and reporting. Fixed a bug in fat function argument splitting.
- January 10, 2003: Numerous changes, including to this manual.
- October 14, 2002: CCured is now available under the BSD license
(see the License section or the file LICENSE).
The correspondence between the source code and the annotated graph can
be found in the infer.c file. You may need to pass –emitinfer to
ccured to see this file. For a base file named FOO.c, the graph
will be shown in FOOinfer.c. This file represents the state of the tool
after pointer kind inference but before run-time checks have been inserted
and before names have been mangled. The infer.c file consists of two
sections: the annotated source code and the inference graph.
A.0.1 Annotated Source
The annotated source code is just the original source code with the special
syntax __NODE(n) associated with every pointer type in the program.
There are two main places where the __NODE attribute can appear:
immediately following a pointer-type constructor * or immediately
following the name of a variable. In the former case the attribute specifies
the node associated with the pointer-type while in the latter
case the attribute specifies the node associated with the address-of the
variable.
For example, the line:
int * __NODE(3) ptr __NODE(4);
int * __NODE(6) * __NODE(7) matrix __NODE(8);
indicates that node 3 in the graph is associated with the pointer variable
ptr and node 4 is associated with the address of ptr. Node 7 is
associated with the top-level pointer in the variable matrix, while node 6
is associated with the inner pointer (e.g., node 6 is associated with the type
of the expression matrix[0]). __NODE(n) is a type qualifier (like
const or restrict) that modifies the pointer type constructor just to
its left.
Consider the following simple test case:
int main() {
int * ptr;
int base;
ptr = &base;
ptr ++;
}
If we run it through CCured's inferencer we will see:
int main() {
int * __NODE(90) ptr __NODE(91);
int base __NODE(92);
ptr = & base;
ptr = ptr + 1;
}
Node 90 is associated with ptr. The additional nodes to the right are
associated with the addresses of variables. Node 91 is associated with
&ptr and node 92 is associated with &base. Since there is an
assignment between ptr and &base, we expect to see an edge between
nodes 90 and 92 in the graph. The graph itself can be found at the end of
the C code near a line like:
/* Now the solved graph (solver) */
A.0.2 Inference Graphs
In order to see the whole graph, you must pass the argument
–emitGraphDetailLevel=3 to Ccured. The graph at the end of
FILEinfer.c is represented as a printout of every node. Each node
description has the following form:
ID : Location (flags) (node this type points to)
K=Kind/Reason T=(base type of this node)
S=(successor edges)
P=(predecessor edges)
So the entry for node 90 (from the example in Section A.0.1) might
look like:
90 : Local(./simple.i.main.ptr).1 (stack,posarith,) ()
K=FSEQ/from_flag T=int
S=
P=92:Cast
The node number is 90, the location field says that this node represents a
local variable named ptr inside the main function of the file
simple.i. The location of a node is a unique name for the pointer type.
Locations use a hierarchical naming scheme is composed of a place
followed by a period and an index within the place. The following places
are used:
-
Glob(var) - this is associated with the global variable var
- Static(file.var) - this is associated with the file-scope variable
var from file file.
- Local(file.func.var) - this is associated with the local variable
var occurring in function func from file file.
- Field(fieldname) - this is associated with a field in a structured
type.
- Anon(nr) - this is associated with an occurrence of a pointer-type
constructor in a cast or in the sizeof expression.
Since each place can contain many occurrences of a pointer-type constructor
(e.g. in a declaration int * * x) we use indices to differentiate between
such occurrences. For a variable place (Glob, Static, Local), the
index 0 is always associated with the node corresponding to the address of the
variable. Then we traverse the type of the variable in depth-first order (for
functions we start with the result and then continue with the arguments) and
we assign indices starting at 1. For Field and Anon we do a similar
traversal.
The flags field is a list of the following values:
| Flag |
Meaning |
| stack |
this pointer may contain a stack address |
| escape |
this value may be assigned through a pointer
and escape to the heap |
| upd |
a write may be performed through this pointer |
| posarith |
this pointer may be subject to positive
pointer arithmetic |
| arith |
this pointer may be subject to arbitrary
pointer arithmetic |
| null |
this pointer may become NULL |
| int |
an int may be cast into this pointer |
| noproto |
this pointer is associated with a function
that is missing a prototype |
| interf |
this pointer is associated with a function
that is part of an interface |
| sized |
this pointer may point into a sized array |
| reach_s |
this pointer may flow into a string |
| reach_q |
this pointer may flow into a SEQpointer |
| reach_i |
this pointer may flow into an INDEXpointer |
In our example, node 90 has the posarith flag because it represents
ptr and ptr++ appears in the code. Node 90 has the stack flag
because &base is a stack address and the assignment ptr = &base
occurs in the code.
Continuing our example, the text K=FSEQ/from_flag tells us that node
the pointer type associated with 90 is FSEQbecause of one of the flags
(in this case, the posarith flag). That is, the type inference has made
node 90 a forward sequence pointer (one that carries its upper bound with
it) because positive pointer arithmetic is performed on it (so we will need
a run-time check to make sure that it stays in bounds).
The text T=int tells us the base type of node 90, that is, node 90 is
associated with a pointer to an int.
The S= and P= lists show successors and predecessors of this node.
In our example, the text P=92:Cast means that node 90 has a predecessor
edge of type Cast from node 92 (the node for &base). That is,
information from node 92 flows into node 90. This is the graph's way of
representing the assignment statement ptr = &base. There are various
sorts of edges in the graph:
| Edge Name |
Meaning |
| Cast |
the value of pred may flow into succ |
| Compat |
pred and succ must have equal types |
| Safe |
if pred is WILD, succ must be WILDas well |
| |
(usually links structures and their fields) |
| Null |
a NULL flows in the direction of the edge |
| Index |
if pred is INDEX, succ must be INDEXas well |
Of these, Cast, Compat and Safe are the most common. Cast
represents casts and assignments, Safe links up fields and structures
so that they have consistent pointer kinds and Compat ensures equality
between underlying pointers. As an example, in the code:
int * __NODE(1) * __NODE(2) a;
int * __NODE(3) * __NODE(4) b;
a = b;
We would see a Cast edge from 4 to 2 and a Compat edge between 1
and 3. The Compat edge ensures that nodes 1 and 3 will end up with the
same pointer kind.
A.1 Type Names
CCured will also rename types that are expanded to contain extra
information used by the run-time checks that ensure safety. For example, a
SEQpointer needs to carry information about its bounds. This information
is stored adjacent to the main pointer, but as a result a SEQpointer is
larger than a normal C pointer. Thus a structure that contains a SEQpointer will be larger than an otherwise-identical structure that contains
a normal C pointer (or a SAFEpointer). In addition, the offsets of any
structure fields that come after the SEQpointer will be different. To
mark these differences and indicate new types, CCured will change the names
of types and structure. CCured does this by prepending strings to the
structure or type name.
| Pointer Kind |
Type Prefix |
| WILD |
wildp_ |
| INDEX |
indexp_ |
| SEQ |
seq_ |
| SEQN |
seq_ |
| FSEQ |
fseqp_ |
| FSEQN |
fseqp_ |
| SAFE |
No associated prefix |
| RWSTRING |
No associated prefix |
| ROSTRING |
No associated prefix |
For example, given a program of the form:
Foo * ptr;
ptr++;
CCured will probably infer that ptr should be an FSEQpointer. The
resulting cured code will have the following form:
typedef struct { /* type made by CCured */
Foo * __FSEQ _p; /* actual pointer */
void * _e; /* end of the allocated region: upper bound */
} fseqp_Foo; /* FSEQ pointer to an int */
fseqp_Foo ptr;
// code for "ptr++;"
If the original code were:
Foo ** ptr;
ptr++; /* forces ptr to be FSEQ */
ptr[0]--; /* forces *ptr to be SEQ */
We would expect the final type of ptr in the cured code to be
fseqp_seq_Foo, because ptr will have the type “(forward sequence
pointer) to (sequence pointer) to Foo.” Note that fseqp_seq_Foo is
not merely a name mangled form of Foo, since both the definition of
Foo and the definition of fseqp_seq_Foo will exist in the final
cured program.
Appendix B A Tour of the Source Code
This section was updated on Janury 2003
./:
configure: a shell script that creates the CCured makefiles by
scanning your system for existing programs (like gcc)
Makefile.cil.in: instructions for building CIL
Makefile.in: instructions for building CCured
Makefile.ocaml: used by Makefile.ccured for the Ocaml part
Makefile.gcc: Included in the above if you use gcc
Makefile.msvc: Included in the above if you use Microsoft Visual C
./src: (ML code)
main: driver for CCured: parses command line arguments
and transforms its input C files
./src/ccured: (ML code)
cure: inserts run-time checks into C code based on
pointer annotations
curesplit: turns multi-word structure pointer representations
into multiple single-word variables (increases
performance by allowing later compilers to make
better optimizations)
curestats: counts the static number of run-time checks
inserted
curechecks: Some run-time checks for CCured
cxxpp: A preprocessor for EDG's output on C++ lowering.
markptr: marks pointers based on their usage so that the
inferencer can pick an efficient representation
markutil: Various utility functions
optim: optimizes the placement of run-time checks
ptrnode: a graph data structure used by the inferencer
poly: handling of polymorphism
seoptim: a symbolic-execution based run-time check
eliminator
solver: an old inferencer
solveutil: support functions common to all inferencers
type: an implementation of physical subtyping; determines
if one C type is a physical subtype of another
typecheck: verify that the pointer representations have been
assigned soundly
unionfind: a support data structure
vararg: handling of variable argument functions
wrappers: handling of wrappers
./bin/: (scripts)
ccured: a drop-in replacement for 'gcc' and for MS 'cl'
./include/: (C header files for use with programs being cured)
ccured_GNUCC.patch: a description of what to patch (modify) in GCC's
standard header files
ccured_MSVC.patch: as above, but for MS VC
gcc_<version>/: the patched files created when ccured_GNUCC.patch
is applied to the files in your /usr/include.
<version> is your version of gcc. These files are
created when you build ccured.
cl_<version>/: as above, but for MSVC.
ccured.h: included before curing
ccuredcheck.h: inline macros for doing run-time checks (included
after curing)
ccuredannot.h: various declarations and macros common to ccured.h and
ccuredcheck.h
*_wrappers.h: wrappers for various functions in the standard library.
For example, we patch stdio.h to #include
stdio_wrappers.h, so that whenever you include stdio.h
in the target program the appropriate wrappers are
brought in as well.
./lib/: (C code for use at runtime)
ccuredlib.c: a library for error handling, wrapper helpers, etc
gc/: boehm-weiser garbage collector
(http://www.hpl.hp.com/personal/Hans_Boehm/gc/)
browser*: browser Javascript code
Note: this section was written on 6/29/01. CCured has changed since
then..
Once you download the sources for a new package, you can run the translator
on it. To create a Makefile target for a package you typically add to
cil/Makefile a target that just invokes make on the package's own Makefile
with the CC variable bound to “ccured –merge”. I'll use the 6/29/01 version
of ftpd as my example.
First we try to run it without our tool involved:
% make ftpd-clean
% cd test/ftpd/ftpd
% make
This succeeds, generating an 'ftpd' binary. In the case of 'ftpd',
running it is slightly complicated:
% ./ftpd -D -d -p 3333
(then in another window)
% telnet localhost 3333
Trying <IP_ADDRESS>...
Connected to localhost.
Escape character is '^]'.
220 madrone.cs.berkeley.edu FTP server (Version 6.5/OpenBSD, linux port 0.3.2) ready.
(etc)
This is (to our way of measuring) success.
Now we try it in 'cil' mode:
% cd cil
% make ftpd-clean
% make ftpd
At the moment, this also works, producing another 'ftpd' binary. We test
it the same way, and rejoice at its success.
Finally, we dare to try it in 'box' mode, meaning the instrumentation
module will be used:
% make ftpd-clean
% make ftpd INFERBOX=4
After crunching for a while, it reports this error (you have to scroll
back a bit to see the right one):
./ls_all.c:1338: Bug: Calling non-wild ioctl with too many args
This is an error from the 'box' module, complaining about what it
perceives to be a type error. If we investigate the named source
line, we see
if(ioctl(1, 0x5413, & win) == 0 && win.ws_col > 0)
confirming that 'ioctl' is involved. Since the *_all.c files are the
output of our tool, and do not themselves #include any files, we can
simply search in this file for ioctl's declaration. We do so, and see
extern int ioctl(int __fd , unsigned long int __request , ...) ;
Hmmm... looks like it was declared to accept any number (>=2) of args, so
this looks like a bug in the 'box' module; it should accept this code,
but it does not.
The next step is to write a tiny C program which calls ioctl (see
test/small2/ioctl.c), and verify it fails the same way
% make scott/ioctl INFERBOX=4
[...]
ioctl.c:9: Bug: Calling non-wild ioctl with too many args
[...]
Yep, same problem. Now we report this to George, since typically he's
much faster at identifying the problem, since he wrote the 'box' module.
In the meantime (waiting for George to magically fix the problem), we
could temporarily comment-out the call so we can proceed to find other
bugs. Or, perhaps we change the ioctl call to instead call a wrapper
function (wrappers are defined in lib/ccuredlib.c, which gets linked into
the translated program).
Eventually (see "make go INFERBOX=4") we'll get an executable. If it
runs correctly, celebration is in order. If not, it will usually fail
because of a failed runtime check (this one is from a test vector for
which go fails);
% make go INFERBOX=4
% cd test/spec/099.go/src
% ./go 5 4
[...]
array bug: index is 5980 (vs 5980)
Failure: Ubound
Abort
Tracking down the source of such failures is the most time-consuming
part of pushing a program through. Sometimes it's a bug in the
translator, in which case ideally a test case can be isolated for easy
diagnosis.
Sometimes (more and more often) it's a bug in the original program (go
had 10 array bounds violation bugs at last count). In this case you
have to change the original code to fix the bug; this may be easy or
hard. If it's hard, try just surrounding the offending statement with
an explicit bounds check in an 'if' statement, so the program skips
the bad statements (that is what I did to cause all of the "array bug"
outputs in the "5 4" case above).
C.2 Writing Wrappers Manually
To interface with external code, you are usually better off using the automatic
wrapper system described in Chapter 8. However if that isn't possible,
you'll need to write a wrapper directly in C:
-
Step 1:
-
Consult the name-mangling algorithm documented Section 8.1
and ccuredlib.c to decode the required types of the parameters.
- Step 2:
-
Determine the semantics of the function being wrapped (e.g.,
if it's a unix libc call, consult its man page). In particular,
find out how memory passed via pointers is accessed (read and
written).
- Step 3:
-
Write the wrapper, and make calls to the verification
and query functions in the section in ccuredlib.c titled
“general-purpose”. If the function manipulates wild pointers,
be sure to update tags; conversely, if no wild pointers are
involved, there are no tags to worry about.
Good examples to consult (in ccuredlib.c) include read_w,
fgets_ffw, stat_ww, strcat_www, memmove_www.
C.3 Apache Modules
This section applies to CCured as of July 2002
C.3.1 Introduction
This writing assumes Apache 1.3.19 and an x86/Linux system. Apache is an
open source web server that has the ability to dynamically load third-party
modules. Modules can examine and alter HTTP requests and also examine and
alter the webserver replies. For example, a compression module might
examine the HTTP request to see if it contains the “Accept-Encoding:
gzip” tag. If it does it might alter the HTTP reply, replacing the text of
the webpage body with a compressed version of that text. Modules can be
configured (via a file called httpd.conf) so that their behavior is
limited to a certain location or directory.
Apache modules share the same address space as the Apache webserver: no
software fault isolation is present. As a result, if the module crashes it
brings down that webserver (although apache is usually configured to
immediately spawn a new webserver thread to replace the fallen one). More
distressingly, a module with a security violation (for example, a
format-string bug) can allow remote users to gain shell access to the
webserver machine (one version of mod_php3 features such a
vulnerability: CCured prevents that vulnerability).
Apache modules are typically single files with a fairly standard naming
convention: mod_foo.c is the foo module, where foo
ranges over fairly descriptive keywords like gzip, random, urlcount, auth or layout. mod_foo.c almost invariable
contains a global data structure of type module with the C name foo_module. This data structure is a table of function pointers and entry
points. Once mod_foo.c has been compiled to the shared object mod_foo.so, Apache will dynamically load it and call the function
pointers listed in the foo_module structure at the appropriate time
(e.g., when a new request comes in).
C.3.2 Curing Apache Modules
Most Apache modules are of a relatively modest size and curing them is no
great chore. However, some annotation work must be done. Since the cured
module must interact with the non-cured Apache webserver, objects that are
passed between them must not change size. As a result, WILD and other
fat pointers cannot be introduced into such objects. Annotations must be
added to convince CCured that the module can be made safe without such
run-time checks.
Imagine that you are trying to cure mod_urlcount.c. Take out your
favorite text editor and open up the file. Near the top you should find a
configuration record structure. Each module defines a separate
configuration record structure with a separate (non-exported) name. For
example, mod_urlcount has:
typedef struct urlcount_config_rec {
int urlcount_default;
CounterType urlcount_type;
int urlcount_auto_add;
char *urlcount_file;
} urlcount_config_rec;
Each module also contains functions that create, manipulate and merge such
configuration structures. This is the mechanism through which Apache modules
maintain global state. Each time Apache calls one of the function pointers
exported by the module, it passes along a way to get to the appropriate
configuration record. Since Apache does not know how the config structure
will be defined, it uses void pointers to describe the type. CCured
comes with a set of macros that instantiate those void *s on a
per-module basis. Add the line:
NEW_MODULE_TYPE(urlcount, urlcount_config_rec) // this is a macro
where the first parameter is the module suffix name and the second is the
type name of the configuration record. This macro declares a type named
module_urlcount. As mentioned earlier, each module exports a module
structure (full of function pointers). We must redeclare the module to take
advantage of the instantiated types. Change:
module urlcount_module; // full of "void *"s
to
module_urlcount urlcount_module; // uses "url_config_rec *", not "void *"
Now scroll down a bit and look for the word keyword void. Apache
modules often feature unnecessary casts to void. For example, mod_headers contains this function:
static void *merge_headers_config(pool *p, void *basev, void *overridesv)
{
headers_conf *a = (headers_conf *) ap_pcalloc(p, sizeof(headers_conf));
headers_conf *base = (headers_conf *) basev,
*overrides = (headers_conf *) overridesv;
a->headers = ap_append_arrays(p, base->headers, overrides->headers);
return a;
}
Every void in this function really stands for headrs_conf (the
mod_headers version of urlcount_config_rec). Change it so
that the void types are no longer present:
static headers_conf *
merge_headers_config(pool *p, headers_conf *basev, headers_conf *overridesv)
{
headers_conf *a = (headers_conf *) ap_pcalloc(p, sizeof(headers_conf));
headers_conf *base = (headers_conf *) basev,
*overrides = (headers_conf *) overridesv;
a->headers = ap_append_arrays(p, base->headers, overrides->headers);
return a;
}
Repeat this process with all configuration functions that contain void. Now search for ap_get_module_config. It is a macro that
contains a (safe) cast to and from void * – it allows modules to
extract their configuration record from the global server state. For
example, mod_headers contains:
headers_conf *serverconf =
(headers_conf *) ap_get_module_config(s->module_config, &headers_module);
Change this to:
headers_conf * serverconf;
{ __NOBOXBLOCK
serverconf = ap_get_module_config(s->module_config, &headers_module);
}
The __NOBOXBLOCK block keyword tells CCured to leave the block
alone: we are asserting that it is already safe. Modify every instance of
ap_get_module_config and ap_get_perdir_module_config
the same way.
Now look for a datatype with the suffix entry. For example, mod_headers features:
typedef struct {
hdr_actions action;
char *header;
char *value;
} header_entry;
This marks a use of Apache's polymorphic (via void *) array routines.
Insert the following macro declaration to tell CCured about this array
type:
NEW_TABLE_TYPE(header_entry, header_entry) // macro
This macro declares a new type, array_header_FOO (where FOO
is the first argument) that is a specialized version of the Apache-provided
type array_header. Other data structure (for example, the
configuration record structure) will contain array_headers. We
change them to use this new datatype. Change all declarations like:
typedef struct {
array_header *headers;
} headers_conf;
into:
typedef struct {
array_header_header_entry *headers;
} headers_conf;
Now search for every call to ap_make_array, ap_append_arrays, ap_push_array and append FOO (in our
running example, header_entry) to the name of each called function.
For example, change:
new = (header_entry *) ap_push_array(dirconf->headers);
into:
new = (header_entry *) ap_push_array_header_entry(dirconf->headers);
Finally, surround all global table declarations with #pragmas that
tell CCured to leave them alone (because Apache must read them). Often
there are three such global tables per module. One is an array of struct const command_recs, one is an array of struct const
handler_recs, and the last is a module. Surround them all with #pragmas as follows:
static const handler_rec mod_gzip_handlers[] =
{
{"mod_gzip_handler", mod_gzip_handler},
{CGI_MAGIC_TYPE, mod_gzip_handler},
{"cgi-script", mod_gzip_handler},
{"*", mod_gzip_handler},
{NULL}
};
becomes:
#pragma box(off)
static const handler_rec mod_gzip_handlers[] =
{
{"mod_gzip_handler", mod_gzip_handler},
{CGI_MAGIC_TYPE, mod_gzip_handler},
{"cgi-script", mod_gzip_handler},
{"*", mod_gzip_handler},
{NULL}
};
#pragma box(on)
Finally, change the declaration of the global module to use the new
specialized type we created earlier. For example, change:
module MODULE_VAR_EXPORT urlcount_module = {
into:
module_urlcount MODULE_VAR_EXPORT urlcount_module = {
Voilą.
C.3.3 Linking Apache Modules
Suppose you have just finished making the source modifications to mod_foo.c. Now you want to test it on Apache. Use CCured to compile it to
mod_foo.o. Make sure that there are no WILD pointers and that
the sizes of types involved in the apache-module interface did not change.
Now you must link it:
$ gcc -shared -o mod_foo.so mod_foo.o
$ cp mod_foo.so /path/to/apache/bin/
Now go to your Apache binary directory and edit httpd.conf. Go to the
LoadModule section and add something appropriate according to the
documentation for your module. For example, mod_usertrack can be
configured by adding:
LoadModule usertrack_module bin/mod_usertrack.so
CookieTracking On
CookieExpires "1 days"
Now try to start Apache:
$ ./apachectl stop
$ ./apachectl start
./apachectl start: httpd started
If you see the “httpd started” message, it worked. If there were
messages about undefined symbols, you probably have to write a few
wrappers. For example, you might see:
./httpd: undefined symbol strdup_ff: mod_foo cannot be loaded
In this case you must write a wrapper for strdup that uses FSEQ
pointers. Suppose you write it in wrapper_foo.c and compile that to
wrapper_foo.o. Now go back to the linking step:
$ gcc -shared -o mod_foo.so mod_foo.o wrapper_foo.o
$ cp mod_foo.so /path/to/apache/bin/
And try to start Apache again. Eventually this process converges (you can
skip ahead by using a utility like nm to list all of the undefined
symbols in mod_foo.so if you like) and your Apache module will be up
and running.
If for some reasons your Apache module crashes at run-time, consider using
the underlying CIL –logcalls mechanism to track down the error
(Apache modules do not treat well with normal debuggers). Make sure that
the debugging comments are directed to syslog(3) rather than printf(3) or somesuch.
As daunting as it may seem, it actually takes less than 30 minutes to Cure
an Apache module of average size and get it up and running. Some of that
time is spend reading the module's documentation so that it can be loaded
and tested correctly. Good luck!
Appendix D Using the Regression Tester
The regression tester is a program that allows you to do two things:
-
Run a list of shell commands and capture their standard and error output
in a log file, and
- Analyze such log files and extract various information, among which the
most important is which test cases have succeeded and which have failed.
- Compare results of two runs according to many parameters, such as
whether the test succeeded or not, how much time CCured took to process it,
how many different kinds of pointers were inferred and how fast is the
resulting program.
Since the running of the tests and the analysis of the output is separated
you can easily do things like compare the results on multiple runs, extract
various reports from a single output (like what tests have succeeded, which
have failed, plus such information split by test groups). You can also extract
some data from each test (such as the running time) and make simple reports.
Test cases can have comments associated with them (such as reminders of why
it fails) and can be associated with zero or more test groups.
D.1 Running the regression
The regression tester is implemented in Perl as "testsafec.pl", which in
turn contains simple wrappers for functions provided by the more generic
RegTest.pm.
The regression tester uses relative paths so it must be run in the
cil/test directory.
The basic command for running the tests is "testsafec –run". This runs
all of the test cases, saving the log in the file "safec.log". Before
creating this file, it renames previous versions of this file as
"safec.log.<n>" where n is an integer starting from 1 to a maximum number
that is configurable.
The following command line options are useful for running the tests (see
"testsafec –help" for a complete list:
--one <testname> : runs only the named test
--gory : shows lots of details about the execution of the
test, such as the commands executed
--dryrun : only pretends to run the test. Useful to see what
would be run
--log : select the base name of the log file (default
"safec.log")
--logversions <n> : keep logs up to version <n>. Default is 5.
--noremake : runs the commands without trying to remake the safec
compiler before each test. Useful if you want to
work on the compiler while the tests are running
--safecdebug : uses the DEBUG version of the safec compiler and
uses the C compiler in debug mode. By default it
used the RELEASE version and the optimizing compiler.
--group <groupname> : adds all the tests in the named group to the list of
tests to be run or to participate in the analysis of
the log. (Right now we have groups: apache,
bad, cil, box, infer.) If no such option is
specified then all tests are selected. Multiple such
options can be given and are cumulative.
--nogroup <groupname> : excludes the tests in the named group from running
or from the analysis. Multiple such options can be
given and are cumulative. These options are
processed after all --group options have
been processed.
--listtests : list the tests that are enabled along with their
group membership. This is useful to find out what
tests and groups exist.
--stoponerror : stop at the first error
--showoutput : show the output on the console. Normally output is
saved in a file.
D.2 Analyzing the results
The basic command for analyzing log files is "testsafec". This will
prompt the user to select one of the several log files that exist in the
current directory and then (by default) it will print a list of the failed
test cases, with a short (user provided) comment and the last error message
detected in the output for that test case.
The following commands are useful during analysis:
--log : select the log file (see above)
--group, --nogroup : select groups (see above)
--listtests : list tests and groups (see above)
--param=<pnames> : show a report about the successes, with the columns
being the named parameters (separated by ,). Run
"testsafec --help" to see what parameters are
available. Use --param=ALL to make a report with
all available parameters.
--sort=<pnames> : sorts the report by the given parameters.
Furthermore, the reports that are generated can be compared. To compare the
results of two runs (whose logs are say “safec.log.1” and “safec.log.2”,
run
test/compare safec.log.2 safec.log.1 --group=slow
This will compare the results of safec.log.2 using as reference the
results in log safec.log.1.
D.3 Configuring the regression
For this you have to edit testsafec.pl. You will see a large section in the
middle of the file containing lines like:
\$TEST->add3Tests("test/array1");
(check out the definition of add3Tests at the bottom of the file). This
adds three tests named "test/array1-cil", "test/array1-box" and
test/array1-inferbox", each one containing one command that invokes
"make test/array1 ...", where ... are appropriate parameters.
A second optional string parameter to add2Tests is something to be added
to the command line.