MinGW x86_64 ABI problem

[John Fine]

Joern Rennecke wrote:
> To a first order of approximation, that's the CALL_USED_REGS macro in
> tm.h (config/i386/i386.h).
>   
Thankyou.
> If a port does
> not anticipate that a certain register class may need saving, the generic
> code to implement saving of registers across calls can emit instructions
> that can not be recognized,
I figured that out after sending the first email.  But I don't know what 
to do about it.

There is no instruction to push or pop an xmm register.  The code seems 
to expect to be able to push and pop.

If I knew more about gcc internals, I could invent a multi instruction 
sequence to push or pop, but the result would be very inefficient 
(because several xmm registers need to be saved and restored in the 
typical function).

There are architectures that don't have push/pop at all as well as some 
where push/pop are slower than the alternative.  In many compilers (I 
assume gcc, but haven't checked) the prolog code can sub the right 
amount from rsp for both local variables and saving 
callee-saved-registers, then move those registers into the right place 
in the stack frame.

What/where do I change in gcc to tell it that is the way to save and 
restore xmm registers.
> You'd have to fix the machine description to recognize and implement properly
> these register saves and restores.  And then you'll have to recompile
> all the libraries with your changed ABI.
>   
I expected I might need to do that.  I'm trying to find out how.
> If you only have a few entry points in your DLL, it might be simpler
> to write some wrapper code in assembler to save and restore the xmm
> registers.
>   
I have considered that.  But it is quite a bit trickier than that 
description implies.  The DLL is actually built on the fly while the 
main application is running.  I don't have a decent way to know which 
entry points even use xmm registers.  It would be quite ugly to save ten 
128 bit registers on every entry point when only a few of those entry 
points even use them.

Any idea why I don't see this issue discussed on the net, and why 
someone else hasn't fixed it already?  Or have they and I just haven't 
looked in the right place?

Doesn't anyone else try to create .lib or .dll files with MinGW that can 
be called from standard win64 .exe's?

As soon as the called function has a few float or double local 
variables, plus any function up the call stack in the exe also has a few 
float or double local variables, the result should malfunction.  I can't 
believe I'm the only one seeing this.