Esthetics (or worse?) of Secure Pointers

[Robert Bernecky <bernecky at acm dot org>]

My apologies for the length of this message...

Thanks to John Gilmore, I'm getting my feet (and other body
parts) wet by picking up Greg McGary's Bounded Pointers work
on GCC, with the goal of improving application security
by preventing programs from violating array/string bounds,
while minimizing performance loss.

There are a few design issues to be dealt with. This message
asks for recommendations on two of them: glibc and bounds-checking
control.

The glibc change is difficult: due to internal changes in
pointer structure, glibc has to be recompiled in toto.
Programs compiled with bounds checking are incompatible
[from a linker/loader standpoint] with existing [non-bounds-chedked]
programs: the function call interface has to change for
pointers.

>From a sanity standpoint, we'd like to be able to
intermix bounds-checked and non-bounds-checked functions.
John, Greg, and I discussed possible resolutions to this problem.
John and Greg may wish to correct me on my appreciation of our
discussion, which follows.

We discarded the function-level intermixing as too low-level
and complicated in practice. This left us with larger approaches to
the problem of intermixing checked and non-checked code:

a. Multiple libraries: discarded as too fragile and
   unstable, at least in their current state.

b. Rebuild entire applications with the new library. John's opinion
   is that, at least in Linux, we have complete source
   code availalbe, so building an entirely new distribution
   is reasonable.
   Obviously, non-Linux GCC users may not have this degree
   of control, but come the revolution... Oops, sorry. 8^}

   This is messy, in my view, as it is be possible,
   in practice, to introduce bounds checking into
   a bazllion lines of code and have it all work.
   So, we'd like to have a NudgeWare(tm) solution, in which
   we can migrate applications in a more controlled manner.

c. (Bear with me...) Greg's work controls bounds checking
   by a compiler flag:  --enabled-bounded.
   This means that we use the same glibc for both checked and
   non-checked output.
   John suggested an alternate approach which may work fine,
   but Greg is a bit leery of it, and I don't know enough about
   the issues: Bounds-checked code would be produced by
   introducing a new target architecture. My understanding of
   this us that it would let us have multiple glibcs lying
   around, in a manner that we could control easily.

   One of our concerns here is that this is like using
   a wrench as a hammer.

So, the question boils down to:

	What is the appropriate way to introduce an incompatible
        glibc, preferably in a way that lets it coexist with
        the existing glibc?

Thanks.
Bob


Robert Bernecky                  Snake Island Research Inc.
bernecky@acm.org                 18 Fifth Street, Ward's Island
+1 416 203 0854                  Toronto, Ontario M5J 2B9 Canada
http://www.snakeisland.com