This is the mail archive of the gcc-patches@gcc.gnu.org mailing list for the GCC project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Add support for the Win32 hook prologue (try 3)


Am Friday 11 September 2009 15:58:09 schrieb Pedro Alves:

The compiler switches, I haven't found them before. I always thought MS 
generates the code with the __naked__ attribute or some other per-function 
attrib.

> From that description, it seems MSFT doesn't do any effort to keep
> the standard 'push ebp, mov esp, ebp' there.  E.g., it sounds like
> (omit frame pointer) /Oy + /hotpatch wouldn't create that
> whole 5-byte sequence.  Makes me wonder if the new gcc attribute
> shouldn't either, but instead just add the 2-byte nop, like MSFT's
> switch; then, when targetting MSFT abi (mingw32, win64 + -m32),
> whenever a stack frame is being built, output:
This would be a pain for our purposes because we need the 5 bytes to be there. 
My first patches indeed just added the 2 byte nop and changed the mov, but 
there were the following problems:

* Windows apps(or well, Steam specifically) don't like the LEA that is part of 
the stack realignment code, and the 2 byte nop isn't enough for them to apply 
the hook

* If the frame pointer is not created for whatever reason, other instructions 
from later on in the functions might slip into the first 5 bytes and break 
the hook. Usual candidates are an xor %eax, %eax or the -fPIC function call.

The problem of copying the microsoft way 1:1 is that there's no guarantee that 
the generated code is hookable by the app - it depends on the combination 
of /hotpatch and the MSFT abi generating the right combinations of functions, 
but if the compiler decides to drop the frame pointer or realign the stack 
things break.

Things do break on Windows every now and then as well. A system hotpatch or a 
rootkit that adds hooks can break things for example, and not all functions 
Steam tries to hook are hotpatchable on Windows, but since Steam has a 
slightly more robust hooker that can deal with a few other opcodes as well it 
can still hook those functions.

I'd propose to have the function attrib the way it is, and add something 
similar to /hotpatch or /functionpadmin later if needed(e.g. reactos or mingw 
might be interested in that, but I don't think Wine will ever need it). That 
way Wine can more reliably generate code that Windows apps know how to hook.

I'd be glad if a gcc maintainer could comment on that, so I know which way to 
go.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]