Re: [PATCH] Add support for the Win32 hook prologue (try 5)

[Stefan Dösinger <stefan at codeweavers dot com>]

Am Tuesday 22 September 2009 23:20:12 schrieb Richard Henderson:
> static_chain_on_stack will only ever be true for nested functions.
> You should not need to apply this attribute to a nested function,
> as such functions are never exported.
>
> You could change handle_abi_attribute to reject this attribute
> for nested functions by checking decl_function_context (*node) == NULL.
This doesn't seem to work. I added this:

if (decl_function_context (*node) == NULL)
  {
    error ("ms_hook_prologue is not compatible with nested functions");
  }

in ix86_handle_abi_attribute, but when I try to compile a function with this 
attribute I get this error:

test.c:2:1: internal compiler error: tree check: expected tree that 
contains 'decl minimal' structure, have 'function_type' in 
decl_function_context, at tree.c:7993
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://gcc.gnu.org/bugs.html> for instructions.

I am also not sure if ms_hook_prologue is not useful to nested functions. I 
can imagine something like the attached test2.c program. I don't know if it 
is valid C, and I also don't know if the function attribute can be used by 
any other language that has better support for nested functions(in C they're 
a gcc extension as I understand).

I think for now I'll allow the attribute on nested functions, and revert the 
mov and pop like I do if no frame pointer is used. I can imagine that the 
pushl.s %ebp can be used instead of the push the ix86_static_chain_on_stack 
generates for stack adjustment. However, Wine won't ever need this so this 
would be an optimization that won't get much real world test coverage.

#include <stdio.h>

void *test(int x)
{
    int a = x;
    void inner()
    {
        printf("nested func, %d\n", a);
    }
    printf("inner: %p\n", inner);

    /* Is it valid to pass a pointer to a nested function? Is accessing stack variables of
     * the outer function valid after the outer function has terminated? */
    return inner;
}

int main()
{
    void (*func)();

    func = test(3);
    printf("%p\n", func);
    func();
    return 0;
}