Re: Add support for the Win32 hook prologue (try 3)

[Pedro Alves <pedro at codesourcery dot com>]

On Friday 11 September 2009 13:20:41, Stefan Dösinger wrote:
> Am Friday 11 September 2009 13:34:03 schrieb Pedro Alves:
> 
> > If the new attribute doesn't add some assembly directive so that the
> > assembler/linker makes sure the slack is there, users of the attribute
> > have to do it manually themselves somehow.  Otherwise, if nothing adds
> > the nops, you may find that Steam, Xfire and others corrupt the end
> > of a previous function that is unfortunate enough to be placed where the
> > nops slack should be.  Did you consider this?
> Yeah, I did.
> 
> The apps that hook the functions replace the first 5 bytes of the function 
> with an absolute jump. Only Microsoft patches the first two bytes with a 
> relative jump, and then places a jump somewhere before the function starts. 
> As such, the win apps don't replace any bytes before the function. That's the 
> reason why my patch makes sure the push %ebp, mov %esp, %ebp are there.

I see.  I found MSFT's "/hotpatch" compiler switch:

  <http://msdn.microsoft.com/en-us/library/ms173507.aspx>

 "When /hotpatch is used in a compilation, the compiler ensures that first
  instruction of each function is two bytes, which is required
  for hot patching."

From that description, it seems MSFT doesn't do any effort to keep
the standard 'push ebp, mov esp, ebp' there.  E.g., it sounds like
(omit frame pointer) /Oy + /hotpatch wouldn't create that
whole 5-byte sequence.  Makes me wonder if the new gcc attribute
shouldn't either, but instead just add the 2-byte nop, like MSFT's
switch; then, when targetting MSFT abi (mingw32, win64 + -m32),
whenever a stack frame is being built, output:

         8b ec     movl.s %esp,%ebp

... instead of:

         89 e5     mov    %esp,%ebp

This seems like it would get rid of things in the patch like
special handling of -fomit-frame-pointer (but maybe not of
stack realigment handling).


... and the "/FUNCTIONPADMIN[:space]" linker switch:

 <http://msdn.microsoft.com/en-us/library/ms173524(VS.80).aspx>

 "space (optional)

    The amount of padding to add to the beginning of each function, 5, 6, or
    16. x86 images require five bytes of padding, x64 images require 6 bytes,
    and images built for the Itanium Processor Family require 16 bytes of
    padding at the beginning of each function.
    ...
    In order for the linker to produce a hotpatchable image, the .obj files
    must have been compiled with  /hotpatch (Create Hotpatchable Image)."

I would guess '/hotpatch' puts something for the linker in the .drectve
section.


Too bad this isn't being implemented fully, as it might turn out
generaly useful, and, I would guess that you may find out later
at some point some app that also checks if the slack nops are
there in addition to the prologue sequence with extra 2-byte nop.


I do understand that what you have already is progress and serves
your purposes.  To be clear, I'm not a gcc maintainer in any way,
and I'm not objecting to what you have.  (And I'm not going to work
on any of this myself either.)


> So far only Microsoft's system patches replace only the two byte nop(which can 
> be replaced while the process is running, since its an atomic instruction). I 
> also don't think Steam & friends get along with MS hotpatches. Afaik if you 
> install Windows updates, the Steam hook functionality is broken until you 
> reboot Windows(or at least restart the game. I don't know when the hot 
> patches are replaced with a freshly loaded DLL)
> 
> Steam takes care about race conditions while patching the 3 opcodes by either 
> patching them before the process starts(process is created in a suspended 
> state), or patching them during DLL load before the functions have any chance 
> of being executed(It hooks the library load functions to archive this).

I see.  It's one jump less at runtime if you can hook at startup time.

> On a partly related topic, I think the Win64 ABI requires that the first 
> function is two bytes long, and there at least 6 bytes of slack before the 
> function. Does gcc implement that?

I have no idea.

> (Currently I know of no Win64 apps that try to do Win64 function hooking that 
> way, so I don't know yet if those apps will pick up Microsoft's intentions or 
> invent their own way like in Win32)

-- 
Pedro Alves