ABI compatibility regression: Return values on x86

[Andrew Haley <aph at redhat dot com>]

gcc (x86) recently changed its behaviour when returning values shorter
than int.  It used to sign extend, and now it doesn't.

short func2( short *size) { return *size; }

trunk:

func2:
        pushl   %ebp
        movl    %esp, %ebp
        movl    8(%ebp), %eax
        movzwl  (%eax), %eax
        popl    %ebp
        ret

gcc, all previous versions:

func2:
        pushl   %ebp
        movl    %esp, %ebp
        movl    8(%ebp), %eax
        movswl  (%eax),%eax
        leave
        ret

This applies to both 32- and 64-bit gcc versions.

This ABI change was caused by 

  svn diff -r126479:126480 svn+ssh://gcc.gnu.org/svn/gcc/trunk

2007-07-09  Richard Guenther  <rguenther@suse.de>

       * c-decl.c (start_function): Do not promote return type.

Index: gcc/c-decl.c
===================================================================
--- gcc/c-decl.c        (revision 126479)
+++ gcc/c-decl.c        (revision 126480)
@@ -6270,18 +6270,6 @@
   declare_parm_level ();
 
   restype = TREE_TYPE (TREE_TYPE (current_function_decl));
-  /* Promote the value to int before returning it.  */
-  if (c_promoting_integer_type_p (restype))
-    {
-      /* It retains unsignedness if not really getting wider.  */
-      if (TYPE_UNSIGNED (restype)
-         && (TYPE_PRECISION (restype)
-                 == TYPE_PRECISION (integer_type_node)))
-       restype = unsigned_type_node;
-      else
-       restype = integer_type_node;
-    }
-
   resdecl = build_decl (RESULT_DECL, NULL_TREE, restype);
   DECL_ARTIFICIAL (resdecl) = 1;
   DECL_IGNORED_P (resdecl) = 1;

This is generic code; I don't think there was any intention to change
the x86 ABI.  

The 32-bit psABI says 

"A function that returns an integral or pointer value places its
result in register %eax. 

"[ ... ] Functions pass all integer-valued arguments as words,
expanding or padding signed or unsigned bytes and halfwords as
needed."

It is not explicit that return values are handled in the same way as
incoming args, but IMO it is reasonable to assume so.  In any case,
we'd have to have a very good reason to change the ABI at this stage.

Ian Taylor pointed out that any change to this wouldn't be visible to
gcc-generated code, which is true.  This is why, I suppose. no-one
noticed it, despite the fact that it's an ABI change.  However, it may
well break other languages that link to gcc.  It certainly caused
libffi test failures, which is how we noticed it.

So, what now?  Can we even agree about what the psABI actually says
about sign-extending result values?  Was what we did before correct,
or what we do now?  I don't believe that it doesn't matter.

Andrew.

-- 
Red Hat UK Ltd, Amberley Place, 107-111 Peascod Street, Windsor, Berkshire, SL4 1TE, UK
Registered in England and Wales No. 3798903