When using gcc in an environment where -mrtd is default calling convention, gcc calls long long operations like __divdi3 without -mrtd (on the other hand, it honours -mregparm when calling these functions). The resulting code crashes, because stack is incremented twice --- once at the end of __divdi3 and next after return to calling function.
This is test code for the bug: long long __divdi3(long long a, long long b) { return a + b; } f() { volatile int x = 1; long long a1 = 4, a2 = 2; /*long long a = __divdi3(a1, a2);*/ long long a = a1 / a2; if (x != 1) abort(); return a; } main() { return f(); } compile it with -fomit-frame-pointer -mrtd flags. When you run it, it fails in Abort function, because value of variable `x' changed because stack pointer is incorrect in function f. When you look at assembler, you see that __divdi3 exits with ret $16 and there's additional addl $16, %esp after calling __divdi3. Mikulas
I can confirm this on the mainline (20030805).
Most likely the same bug as PR 12769.
There's a similar issue with trying to override certain builtins calling convention with -mrtd in effect. Consider extern void* __attribute__((__cdecl__)) memcpy(void*, void*, __SIZE_TYPE__); we even after seeing this decl emit calls to memcpy with -mrtd ABI out of emit_block_move_via_libcall.
*** Bug 22017 has been marked as a duplicate of this bug. ***
Author: ktietz Date: Mon Apr 11 17:44:04 2011 New Revision: 172268 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=172268 Log: 2011-04-11 Kai Tietz <ktietz@redhat.com> PR target/9601 PR target/11772 * config/i386/i386-protos.h (ix86_get_callcvt): New prototype. * config/i386/i386.c (ix86_handle_cconv_attribute): Adjust comment. (ix86_is_msabi_thiscall): Removed. (ix86_is_type_thiscall): Likewise. (ix86_get_callcvt): New function. (ix86_comp_type_attributes): Simplify check. (ix86_function_regparm): Use ix86_get_callcvt for calling convention attribute checks. (ix86_return_pops_args): Likewise. (ix86_static_chain): Likewise. (x86_this_parameter): Likewise. (x86_output_mi_thunk): Likewise. (ix86_function_type_abi): Optimize check for types without attributes. * config/i386/i386.h (IX86_CALLCVT_CDECL, IX86_CALLCVT_STDCALL, IX86_CALLCVT_FASTCALL, IX86_CALLCVT_THISCALL, IX86_CALLCVT_REGPARM, IX86_CALLCVT_SSEREGPARM): New macros to represent calling convention by flag-values. (IX86_BASE_CALLCVT): Helper macro. * config/i386/netware.c (i386_nlm_maybe_mangle_decl_assembler_name): Use ix86_get_callcvt for calling convention attribute checks and avoid symbol-decoration for stdcall in TARGET_RTD case. * config/i386/winnt.c (i386_pe_maybe_mangle_decl_assembler_name): Likewise. (gen_stdcall_or_fastcall_suffix): Adjust ident and use DECL_ORIGIN for declaration. Modified: trunk/gcc/ChangeLog trunk/gcc/config/i386/i386-protos.h trunk/gcc/config/i386/i386.c trunk/gcc/config/i386/i386.h trunk/gcc/config/i386/netware.c trunk/gcc/config/i386/winnt.c
Fixed.