Bug 44975 - Stack trashed by call inside inline asm
Summary: Stack trashed by call inside inline asm
Alias: None
Product: gcc
Classification: Unclassified
Component: c (show other bugs)
Version: 4.4.3
: P3 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
Depends on:
Reported: 2010-07-18 04:34 UTC by Ramiro Polla
Modified: 2010-07-18 22:08 UTC (History)
1 user (show)

See Also:
Target: x86_64-linux-gnu
Known to work:
Known to fail:
Last reconfirmed:


Note You need to log in before you can comment on or make changes to this bug.
Description Ramiro Polla 2010-07-18 04:34:42 UTC
When there are no functions being called by the C code, gcc decides to use negative offsets in the stack instead of sub'ing rsp. But if there's a call inside an inline asm, it will trash the "stack" with a return pointer, and that might lead to random code being run, for example:
$ cat main.c
void __attribute__((noinline)) dummy(void) {}
void __attribute__((noinline)) runtwice(void (**x)(void))
    void (*f)(void) = x[0];
    asm("movq $0x00, -16(%rsp)\n\t");
#ifdef ONE_IN_C
    asm("call *%0\n\t"::"m"(f));
    asm("call *%0\n\t"::"m"(f));
int main()
    void (*f[])(void) = { dummy };
    return 0;
$ gcc -O2 -g -o main-noc.o -c main.c
$ gcc -O2 -g -o main-c.o -c main.c -DONE_IN_C
$ gcc -o main-noc main-noc.o
$ gcc -o main-c main-c.o
$ ./main-noc
Segmentation fault
$ ./main-c

The disassembly of the runtwice function gives:
0000000000000010 <runtwice>: (no C functions being called)
  10:	48 8b 07             	mov    (%rdi),%rax
  13:	48 89 44 24 f8       	mov    %rax,-0x8(%rsp)
  18:	48 c7 44 24 f0 00 00 	movq   $0x0,-0x10(%rsp)
  1f:	00 00 
  21:	ff 54 24 f8          	callq  *-0x8(%rsp)
  25:	ff 54 24 f8          	callq  *-0x8(%rsp)
  29:	c3                   	retq   
0000000000000010 <runtwice>: (one C function being called)
  10:	48 83 ec 18          	sub    $0x18,%rsp
  14:	48 8b 07             	mov    (%rdi),%rax
  17:	48 89 44 24 08       	mov    %rax,0x8(%rsp)
  1c:	48 c7 44 24 f0 00 00 	movq   $0x0,-0x10(%rsp)
  23:	00 00 
  25:	ff d0                	callq  *%rax
  27:	ff 54 24 08          	callq  *0x8(%rsp)
  2b:	48 83 c4 18          	add    $0x18,%rsp
  2f:	c3                   	retq   

I put that asm("movq $0x00, -16(%rsp)\n\t"); there to explicitly clear the next pointer that would be called, but it could contain anything on a more complex program.

I did not find in the documentation whether it is possible to hint the inline asm block that a function is being called inside it.
Comment 1 Andrew Pinski 2010-07-18 04:40:33 UTC
Well this is called the red zone as GCC assumes anything that does not call a function is a leaf function.
Comment 2 Ramiro Polla 2010-07-18 05:14:36 UTC
Thanks. Is there anything that can be done in the code to say that is not a leaf function, or at least to tell gcc that a call is being made inside the inline asm?
Comment 3 Jakub Jelinek 2010-07-18 13:31:37 UTC
You need to in the inline asm subtract 128 from rsp, do the call, then add it back.
Comment 4 Ramiro Polla 2010-07-18 22:08:50 UTC
That won't work in this case. If I subtract anything from rsp inside the inline asm, the local variable relative to f will no longer be valid. Notice where gcc put it:
  21:   ff 54 24 f8             callq  *-0x8(%rsp)
Is there no way to tell gcc that either a call is being made inside the inline asm or that this function should not be considered a leaf function? If there isn't one yet, could such a feature request be acceptable? (like __attribute__((no_leaf_function)) or something)