gcc stack-smashing protector (for gcc-ss-20001101)

[Hiroaki Etoh]

At 03 Nov 2000 10:59:12 -0800,
 Geoff Keating <geoffk@geoffk.com> wrote:
> > This patch introduces -fstack-protector option, which is a stack-smashing 
> > protection mechanism to the latest snapshot (gcc-ss-20001101).
> > This patch and new files (protector.h and protector.c) are bootstraped
> > and tested on i686-pc-linux-gnu and powerpc-ibm-aix4.4.0.
> 
> This seems to be a large and complex patch for a feature of limited
> usefulness which is already implemented by gcc in a more general form
> (using the full bounds-checking implementation).

This function makes the application hardened against stack-smashing attacks,
which are not yet known by anyone.  Since you can't get rid of all buffer
overflow bugs from the source code of an application at the debugging process,
run-time protection, its overhead should be low, is a practical solution. My
first objective is to provide the protection method that has lowest overheads.

An attacker tends to attack the weakest portion of a system. The targets of
many stack-smashing attacks are changing the return address of vulnerable
function because it is the easiest target. If the return address would be
protected by any run-time protection method, his next target should be
function pointers in local variables or function arguments. For example,
SuperProbe is an utility to examine the type of display devices for X window
system. The exploit code changes the argument variable that refers the
structure with a function pointer to the pointer value that refers the
address of the address of the attacker's code. My second objective is to
eliminate future risks derived from the buffer-overflowed corruption.


> Could you describe in more detail how this was designed, in particular
> what alternatives were considered?

It changes the order of local variables to separate the location of
character buffer and the other type of variables. The decision which
memory reference is related to character variable or not is made from
the address of each reference. I assume that every variables on stack is
pointed by the combination of the start address and its offset.  Since
EXPAND_SUM directive of the expand_expr function doesn't preserve this
characteristics, I changed the use of such directives as follows:
(flag_propolice_protection?EXPAND_NORMAL:EXPAND_SUM)

It copies argument to pseudo register instead of copying to the stack
region as a local variable. From the implementation point of view,
copying as a local variable is easy and simple. But I'd like to minimize
the number of memory reference for the reduction of protection
overhead. The modification of cse.c and gcse.c is required not to
eliminate the pseudo register and not to replace it to the original
argument variable. I introduced the use of volatil bit of SET rtl for
that purpose, which represents the destination register should be alive
after cse and gcse function.


I'd like to contribute to maintain the code related to this protection
as much as possible. Please tell me the next step to put it to the CVS.


Hiroaki Etoh,
Tokyo Research Laboratory, IBM Japan