Stack overflow with stdcall and transfer

[Tobias Burnus]

John McFarland wrote:
> This is one of the reasons that I wanted to simplify this down to a
> reduced test case -- in order to try and isolate the problem.  It
> sounds like you may be right about memcpy creating the problem.  Would
> that indicate a bug in gcc?

I don't think that it is a bug: It is the user's responsibility to use
always the correct calling convention. I am not really a Windows /
target expert, but I think using -mrtd also changes the calling
convention used for calls to the libraries. And if a library such as the
standard C library ("libc") - which contains the memcpy function and
also the gfortran runtime library is not compiled with STDCALL, no one
cleans up the arguments and the stack grows.


> As an aside, I had run into a similar problem a while back when I used
> to use a different open source Fortran compiler, and I put together
> this test case:
>
> PROGRAM prog_orig
>   INTEGER :: n
>   n = 1e6
>   CALL sub(n)
> CONTAINS
>   SUBROUTINE sub(n)
>     INTEGER, INTENT(in) :: n
>     REAL*8, ALLOCATABLE :: x(:,:)
>     INTEGER :: i
>     ALLOCATE( x(n,4) )
>     DO i=1,n
>       x(i,:) = [1d0,2d0,3d0,4d0]
>     END DO
>   END SUBROUTINE sub
> END PROGRAM prog_orig
>
> Now I don't happen to get the crash with gfortran with this test case,
> but the cause may be similar, as it looks like this one should work
> (even with -mrtd) also (assuming ALLOCATE doesn't fail), since memory
> allocation for x would be on the heap.

The problem is *not* the memory of "x" the problem is the memory used
for the function calls. If you want to see which calls gfortran does,
call it using the -fdump-tree-original options. For the latter program,
one finds calls to __builtin_malloc, __builtin_free, and sub. (As "sub"
is in the same file, it uses "stdcall" and properly cleans the stack.)
But as they are only called once, it does not really matter that the
memory on the stack is not cleaned up.

For your original program you had 1000000 times the call to
__builtin_memcpy. memcopy takes two arguments - one pointer and one
size_t, i.e. 2*4*1000000 bytes = 7.6 MB. I think the default stack size
on Windows is 2 MB - thus one can expect a stack overflow.


>> The reason is that -mrtd affects all functions - including those which
>> do not expect STDCALL. With CDECL (the default) - the caller pops the
>> arguments from the stack. With STDCALL the called functions pops the
>> arguments. Thus, if you call with STDCALL a procedure which is actually
>> a CDECL function, the arguments are left on the stack - and you run out
>> of stack space.
>>
>> In general, it is better to avoid -mrtd and annotate only those
>> functions which need it with STDCALL, cf.
>> http://gcc.gnu.org/onlinedocs/gfortran/GNU-Fortran-Compiler-Directives.html
>>
>
> Actually the real problem that is motivating this is that I have set
> up an Excel spreadsheet that uses Visual Basic to interface with a
> large library of Fortran code.  VB requires stdcall.  The reason I am
> using a blanket -mrtd for everything is that I got crashes when I
> didn't compile all source files with -mrtd, and I assumed that it was
> because you can't mix calling conventions (at least for functions that
> call each other?)

Well, you can't mix them - but you already mix them when you call -mrtd.
If you use a stdcall to call a cdecl function, you only leave garbage in
the stack; if you cdecl call a stdcall function, you remove the
arguments twice from the stack which causes more serious problems. But
the real solution is to use the correct calling convention.

For instance (written without testing, hopefully without typos):

module myStdCallFuncs
  use iso_c_binding
  implicit none
contains
  subroutine foo (n, x) bind(C, name='C_name')
!GCC$ ATTRIBUTES  STDCALL :: foo
     integer(c_int), VALUE :: n
     real(c_double) :: x(n)
     ! ... do something
  end subroutine foo
end myStdCallFuncs

That way, you have created a function which has the C prototype
  void __stdcall C_name(int n, double *x );

but is still callable from Fortran using
  use myStdCallFuncs
  call foo(2, [1.0, 2.2])

In that case all functions are CDECL - except for the one ("foo" alias
"C_name") which are explicitly marked as having stdcall. The attribute
also takes care of the typical decoration of Windows's stdcall, i.e. the
suffix @<n> where <n> is the byte size of the arguments.

> For example, say I have 10 API functions defined in the Fortran code
> that I want to call from Excel/VBA, but these API functions make calls
> to hundreds of other Fortran routines that are getting compiled into a
> DLL.  In this case, what would be the preferred approach for setting
> stdcall?  I have been afraid that only setting stdcall for the API
> functions will cause problems since they need to make calls into other
> functions.

There is no problem if the other functions know that they should do a
stdcall. This information is available if you use, e.g., a module as in
the example above.

The directives are described in
http://gcc.gnu.org/onlinedocs/gfortran/GNU-Fortran-Compiler-Directives.html

The C binding is described in
http://gcc.gnu.org/onlinedocs/gfortran/Interoperability-with-C.html -
and that part is compiler independent while the compiler directives part
is vendor dependent. Other vendors have similar directives (e.g. "!DEC$").

Tobias