Re: x86_64 calling conventions and stack frames

[Amittai Aviram <amittai dot aviram at yale dot edu>]

On Dec 24, 2011, at 12:18 PM, Bob Plantz wrote:

> On 12/23/2011 11:37 PM, Amittai Aviram wrote:
>> I am trying to make sense out of the executable code that GCC (4.4.3) is generating for an x86_64 machine running under Ubuntu Linux.  In particular, I don't understand how the code keeps track of stack frames.  In the old days, in 32-bit code, I am accustomed to seeing this as a "prologue" in just about every function:
>> 
>> push %ebp
>> movl %esp, %ebp
>> 
>> Then, at the end of the function, there would either be
>> 
>> sub $xx, %esp   # Where xx is a number based on GCC's accounting.
>> pop %ebp
>> ret
>> 
>> or simply
>> 
>> leave
>> ret
>> 
>> which accomplishes the same thing:
>> - Set the Stack Pointer to the top of the current frame, just below the return address
>> - Restore the old Frame Pointer value.
>> 
>> In 64-bit code, as I see it through an objdump disassembly, many functions do not follow this convention--they do not push %rbp and then save %rsp to %rbp, How does a debugger like GDB build a backtrace?
> 
> Are you looking at leaf functions? They are allowed to use the "red zone" as their stack frame, with rsp as the reference point. Since there are twice as many registers in 64-bit mode, you will find many fewer pushes and pops. Most arguments (the first six integer) are passed in registers and simply moved to the stack frame for safe keeping.
> 
>> x86-64
>> 
>> My real goal here to is to try to figure out a reasonable address to consider as the top (highest address) of the user stack when execution reaches the start of a function further into the program, where perhaps the Stack Pointer has moved down.  I had at first thought that I could use the old backtrace method: chasing saved Frame Pointer values until the value saved is 0--then, the next one after that can count as the highest practical value.  Now, I don't know how to get the equivalent address in 64-bit code.
> 
> If you don't have it, you should get a copy of "System V Application Binary Interface AMD64 Architecture Processor Supplement." It was available at www.x86-64.org, but I can't seem to connect to them right now. The latest version I have is 0.99 (Dec. 7, 2007). It describes the red zone, etc.
> 
> --Bob


Thanks, Bob.  No, I want to get this information from a function that is neither a root nor a leaf, but an internal node of the call graph.  I would like to have a reliable way of finding out where the Stack Pointer is anywhere above the start of the stack frame for main, but below the actual top of the stack, and preferably as close to the top of the stack frame for main as possible.  I still haven't found a way to do this.  Again, I am wondering how people build backtrace routines for x86_64.  (The standard C/Unix library routine backtrace will not give me what I need, because it gives me the return addresses rather than stack frames.  As convention goes, the former addresses will be far lower than the latter.

I have not read that document, and I am having the same trouble that you had with the x86-64.org link, but I've managed to find a copy of the document at 

http://math-atlas.sourceforge.net/devel/assembly/abi_sysV_amd64.pdf

That draft is dated 2005.  Do you happen to know whether that's sufficiently current?

Amittai Aviram
PhD Student in Computer Science
Yale University
646 483 2639
amittai.aviram@yale.edu
http://www.amittai.com