When there are no functions being called by the C code, gcc decides to use negative offsets in the stack instead of sub'ing rsp. But if there's a call inside an inline asm, it will trash the "stack" with a return pointer, and that might lead to random code being run, for example: $ cat main.c void __attribute__((noinline)) dummy(void) {} void __attribute__((noinline)) runtwice(void (**x)(void)) { void (*f)(void) = x[0]; asm("movq $0x00, -16(%rsp)\n\t"); #ifdef ONE_IN_C f(); #else asm("call *%0\n\t"::"m"(f)); #endif asm("call *%0\n\t"::"m"(f)); } int main() { void (*f[])(void) = { dummy }; runtwice(f); return 0; } $ gcc -O2 -g -o main-noc.o -c main.c $ gcc -O2 -g -o main-c.o -c main.c -DONE_IN_C $ gcc -o main-noc main-noc.o $ gcc -o main-c main-c.o $ ./main-noc Segmentation fault $ ./main-c $ The disassembly of the runtwice function gives: 0000000000000010 <runtwice>: (no C functions being called) 10: 48 8b 07 mov (%rdi),%rax 13: 48 89 44 24 f8 mov %rax,-0x8(%rsp) 18: 48 c7 44 24 f0 00 00 movq $0x0,-0x10(%rsp) 1f: 00 00 21: ff 54 24 f8 callq *-0x8(%rsp) 25: ff 54 24 f8 callq *-0x8(%rsp) 29: c3 retq 0000000000000010 <runtwice>: (one C function being called) 10: 48 83 ec 18 sub $0x18,%rsp 14: 48 8b 07 mov (%rdi),%rax 17: 48 89 44 24 08 mov %rax,0x8(%rsp) 1c: 48 c7 44 24 f0 00 00 movq $0x0,-0x10(%rsp) 23: 00 00 25: ff d0 callq *%rax 27: ff 54 24 08 callq *0x8(%rsp) 2b: 48 83 c4 18 add $0x18,%rsp 2f: c3 retq I put that asm("movq $0x00, -16(%rsp)\n\t"); there to explicitly clear the next pointer that would be called, but it could contain anything on a more complex program. I did not find in the documentation whether it is possible to hint the inline asm block that a function is being called inside it.
Well this is called the red zone as GCC assumes anything that does not call a function is a leaf function.
Thanks. Is there anything that can be done in the code to say that is not a leaf function, or at least to tell gcc that a call is being made inside the inline asm?
You need to in the inline asm subtract 128 from rsp, do the call, then add it back.
That won't work in this case. If I subtract anything from rsp inside the inline asm, the local variable relative to f will no longer be valid. Notice where gcc put it: 21: ff 54 24 f8 callq *-0x8(%rsp) Is there no way to tell gcc that either a call is being made inside the inline asm or that this function should not be considered a leaf function? If there isn't one yet, could such a feature request be acceptable? (like __attribute__((no_leaf_function)) or something)