This is the mail archive of the gcc-help@gcc.gnu.org mailing list for the GCC project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Constraints to prevent stack smashing


Hi,

I'm not sure if this is a problem with gcc or merely a lack of
understanding on my part correctly understanding inline assembler
constraints. Consider the following C++ test case:

int main(char* argv, int argc)
{
    unsigned char result;
    void *eflags;

    setIfCondition_Zero(result,eflags);
}

void setIfCondition_Zero(unsigned char &condStatus, void *eflags)
{
  __asm__ volatile      
  (
         "pushf      \n\t"  //pushfdl on 32 bit mode      
         "push %1    \n\t"      
         "popf       \n\t"  //popdfl on 32 bit mode      
         "setz  %0 \n\t"     
         "popf       \n\t"      
         :  "=r"(condStatus): "r"(eflags): "%rsp");   
}

And the generated code for the setIfConditionZero function:

000000000040059c <_Z19setIfCondition_ZeroRhPv>:
  40059c:       55                  push   %rbp
  40059d:       48 89 e5            mov    %rsp,%rbp
  4005a0:       48 89 7d f8         mov    %rdi,0xfffffffffffffff8(%rbp)
  4005a4:       48 89 75 f0         mov    %rsi,0xfffffffffffffff0(%rbp)
  4005a8:       48 8b 45 f0         mov    0xfffffffffffffff0(%rbp),%rax
  4005ac:       9c                  pushfq
  4005ad:       50                  push   %rax
  4005ae:       9d                  popfq
  4005af:       0f 94 c2            sete   %dl
  4005b2:       9d                  popfq
  4005b3:       48 8b 45 f8         mov    0xfffffffffffffff8(%rbp),%rax
  4005b7:       88 10               mov    %dl,(%rax)
  4005b9:       c9                  leaveq
  4005ba:       c3                  retq

The problem should be obvious. The temporary store of the reference gets
stamped on by the stack manipulation. In larger functions containing
inline assembler and calls to subroutines I have seen gcc generate a
stack frame somewhat like:

	push   %rbp
	mov    %rsp,%rbp
	push   %r12
	push   %rbx
==>	sub    $0x30,%rsp
        mov    %rdi,0xffffffffffffffe8(%rbp)
        mov    %sil,0xffffffffffffffe7(%rbp)
        ..
        ..

Making sure that the temporary space isn't stepped on. 

I have worked around the problem for the time being by adding a "sub
0x10, %%rsp" to the inline assembler on the assumption the final leave
will clean everything up. However shouldn't there be a constraint that I
can use to tell gcc the program will use n bytes of stack? I've read the
manual but the section on constraints reads a little too cryptically for
me.

BTW, "g++ --version" == g++ (GCC) 3.3.5

Regards,

--
Alex, homepage: http://www.bennee.com/~alex/
"Nuclear war would really set back cable." -- Ted Turner


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]