Bug 64384

Summary: mingw-w64: stdcall function returning an aggregate is incompatible with MS ABI
Product: gcc Reporter: mity <mity>
Component: targetAssignee: Not yet assigned to anyone <unassigned>
Status: UNCONFIRMED ---    
Severity: normal CC: jacek, jay.krell, ktietz, mizvekov, sezeroz, sven.koehler
Priority: P3    
Version: 4.9.2   
Target Milestone: ---   
Host: Target: i686-w64-mingw32
Build: Known to work:
Known to fail: Last reconfirmed:

Description mity 2014-12-23 13:39:07 UTC
When using COM interface (i.e. calling stdcall function/method), aggregates are returned through an implicit parameter after the this/interface pointer, according to MS ABI. However it is not so when using gcc.

This issue manifests itself when e.g. calling ID2D1RenderTarget::GetSize() from <d2d1.h> as discussed here:


Wine team encountered the issue too, they seem to "solve" the issue on their side by ugly hacks and changing prototypes of the relevant functions for now:

Comment 1 Jacek Caban 2014-12-23 13:55:47 UTC
FWIW the issue is not specific to COM interfaces. It affects all stdcall functions.
Comment 2 Kai Tietz 2014-12-23 14:42:34 UTC
The attribute 'callee_pop_aggregate_return' should fix this issue for you.
This is a general difference between sysv and ms ABIs.  It has in general nothing to do with the stdcall-attribute.
Comment 3 daniel.c.klauer 2014-12-30 23:09:46 UTC
It seems there is a difference between gcc and ms with regards to return in register for C++ methods (regardless of stdcall/cdecl/thiscall).

Tested with i686-w64-mingw32 gcc 4.9.2 and cl.exe from VS 2010:

struct A {
	int field1;

struct A getA(void) {
	struct A a = {123};
	return a;

C function: both gcc and ms return in register (struct is small enough), ok.

struct A {
	int field1;

class Test {
	A getA();

A Test::getA() {
	struct A a = {123};
	return a;

C++ method: ms returns in temp var on stack (as for big structs), but gcc returns in register. 

The same happens for struct containing two floats (as in the original crash issue reported on mingw-w64 mailing list).

Maybe (on 32bit) ms never uses return in register for C++ methods with aggregate result?
Comment 4 mity 2014-12-30 23:45:03 UTC
Daniel, COM interface should be, by definition, language agnostic. COM can be called from C++ as well as from C, and also COM object may be implemented in C++ as well as C. This implies that (at least for stdcall, as COM uses stdcall convention) there shouldn't be any difference between C and C++.
Comment 5 daniel.c.klauer 2014-12-31 16:24:58 UTC
I also think that COM is supposed to be usable from C and C++, but it seems that here we have an exception to this rule.

Let's examine asm generated by the ms compiler for calls to GetSize(), C and C++ versions respectively:

  ID2D1HwndRenderTarget *hwndRenderTarget;
  D2D1_SIZE_F size;
  size = hwndRenderTarget->GetSize();
Code generated by cl:
  lea   edx, DWORD PTR $T76363[ebp]                    # temp var for the result
  push  edx                                            # hidden pointer arg
  [...push THIS arg and get function pointer from vtable into eax...]
  call  eax                     # returns pointer to temp var in eax
  mov   ecx, DWORD PTR [eax]    # reading result from temp var, 1st member
  mov   edx, DWORD PTR [eax+4]  # 2nd member of D2D1_SIZE_F
  mov   DWORD PTR _size$[ebp], ecx
  mov   DWORD PTR _size$[ebp+4], edx

  ID2D1HwndRenderTarget *hwndRenderTarget;
  D2D1_SIZE_F size;
  size = hwndRenderTarget->lpVtbl->Base.GetSize(
      (ID2D1RenderTarget *)hwndRenderTarget);
Code generated by cl:
  [...push THIS arg and get function pointer into ecx...]
  call  ecx
  mov   DWORD PTR _size$[ebp], eax    # reading result from edx:eax registers
  mov   DWORD PTR _size$[ebp+4], edx

The C++ version works well; the C version crashes at the GetSize() call. They're using incompatible aggregate return mechanisms (despite stdcall), but since the d2d1.dll (which, I assume, implements this interface) will only use one of the two mechanisms, the other won't work. Obviously d2d1.dll uses the C++ version because that's what works.

Assuming this is part of MS ABI and not a bug in the MS compiler, then the problem with gcc is that its aggregate return for C++ methods differs from MS ABI (though it's correct for normal C-style functions).
Comment 6 Jay 2018-12-04 11:47:59 UTC
This is probably a bug in Visual C++ when compiling as C.
Or rather, whatever is/was correct, C and C++ should have matched.
At this point, one must be considered correct, the other incorrect, and neither changable.

Since most "COM" programming is done with C++, given no good choice here, the C++ behavior should be considered "correct", and either g++ but not gcc, or  g++ and gcc should follow it. Certainly, at least, g++ should follow Visual C++ C++ behavior.

Consider warning.

Really there is no perfect fix here.

The relatively few C programmers using COM, will have to write a little bit of C++ to interface correctly with such rare functions.
Comment 7 Matheus Izvekov 2020-06-15 02:20:29 UTC
So this is still a problem in GCC 10.1.

Per https://docs.microsoft.com/en-us/cpp/build/x64-calling-convention?view=vs-2019
What GCC does would be correct according to that spec. But in practice, MSVC does different, and windows DLLs expect what MSVC does.

So I have stumbled on this problem with D3D12, calling GetCPUDescriptorHandleForHeapStart method on ID3D12DescriptorHeap object.

Here is definition of this method from SDK:

virtual D3D12_CPU_DESCRIPTOR_HANDLE STDMETHODCALLTYPE  GetCPUDescriptorHandleForHeapStart( void) = 0;

Returned aggregate is just a boxed SIZE_T:

So it should be possible to return on register, but instead here is code from that method (MASM syntax, taken from D3D12.dll):
00007FF83AD6A3F0  mov         rax,qword ptr [rcx+0D0h]  
00007FF83AD6A3F7  mov         qword ptr [rdx],rax
00007FF83AD6A3FA  mov         rax,rdx  
00007FF83AD6A3FD  ret