This is the mail archive of the
gcc-patches@gcc.gnu.org
mailing list for the GCC project.
Re: [PATCH] Omit frame pointer and fix %ebp by default on x86 (take3)
- From: Roger Sayle <roger at eyesopen dot com>
- To: Florian Weimer <fw at deneb dot enyo dot de>
- Cc: gcc-patches at gcc dot gnu dot org
- Date: Thu, 19 Aug 2004 12:03:00 -0600 (MDT)
- Subject: Re: [PATCH] Omit frame pointer and fix %ebp by default on x86 (take3)
On Thu, 19 Aug 2004, Florian Weimer wrote:
> I think it's possible to rewind the stack from the ground up once you
> can read it, so I doubt -fomit-frame-pointer buys you anything from a
> security perspective.
Unfortunately (or fortunately) not. Indeed, I attempted this in the
"plan B" referred to in a previous posting. The proposal was to traverse
the stack looking for stack frames, hoping to identify return addresses
based upon their values, and that they return to a mapped executable
page immediately following a call instruction, and reside in the stack
at a suitable offset from PREFFERED_STACK_BOUNDARY, giving priority to
return addresses adjacent to potential frame pointers, backpointing
stack references. I'd even considered used a dynamic programming
(viterbi) based threading of stack frames, generating the most probable
list of frames (by piecing together their individuals probability).
The major Achilles heel to this approach (which works reasonably on
small testcases) is the large "uninitialized" holes left in the stack.
After running of a while, an x86 stack contains several perfectly
preserved "false" stack frames that confuse stack interpretation.
This problem is analogous to the "pseudo-gene" problem in bioinformatics
where genomic DNA stretches that look like they should make proteins are
actually long discarded relics of evolution.
Of course, programming languages that completely initialize the
contents of their stack frame on each function invocation may be
ammenable to such a technique.
Roger
--