[Bug target/62281] New: gcc doesn't conform to Solaris 32-bit ABI by expecting 16-byte stack alignment

["justin.carveley at oracle dot com" <gcc-bugzilla at gcc dot gnu dot org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=62281

            Bug ID: 62281
           Summary: gcc doesn't conform to Solaris 32-bit ABI by expecting
                    16-byte stack alignment
           Product: gcc
           Version: 4.9.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: target
          Assignee: unassigned at gcc dot gnu.org
          Reporter: justin.carveley at oracle dot com

Created attachment 33400
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=33400&action=edit
Example 2: SSE vector load causes SEGV on callback from Xt timer

This is essentially a resubmission of bug 47842 "gcc forces 16-byte stack
alignment on Solaris i386, when SYSV requires word alignment".

As described in the above bug, changes to gcc for Linux x86 cause generated
code to assume a 16-byte preferred and incoming stack alignment. However, this
behaviour does not conform to the Solaris x86 (32-bit) ABI, which is still
based on http://www.sco.com/developers/devspecs/abi386-4.pdf and only specifies
that the stack be 32-bit word aligned.

PR 47842 was closed due to a lack of test cases. Here, therefore, are two test
cases that demonstrate the problem:


Example 1: Mixing 32-bit modules compiled with the Solaris Studio 12.3 and gcc:

Using:
- Solaris Studio compiler from
http://www.oracle.com/technetwork/server-storage/solarisstudio/downloads/index-jsp-141149.html
- gcc 4.8.0 (stable) or gcc 4.9.0 (unstable) package from
http://www.opencsw.org/
  or gcc 4.9.1 built from tarball

Files: x.c, y.c, build

Result:

GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i386-pc-solaris2.10".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /export/home/justin/tmp/xy...done.
(gdb) r    
Starting program: /export/home/justin/tmp/xy 

Program received signal SIGSEGV, Segmentation fault.
0x080509db in _mm_set_epi8 (__q00=1 '\001', __q01=1 '\001', __q02=1 '\001', 
    __q03=1 '\001', __q04=1 '\001', __q05=1 '\001', __q06=1 '\001', 
    __q07=1 '\001', __q08=1 '\001', __q09=1 '\001', __q10=1 '\001', 
    __q11=1 '\001', __q12=1 '\001', __q13=1 '\001', __q14=1 '\001', 
    __q15=1 '\001')
    at
/opt/tarantella/lib/gcc/i386-pc-solaris2.10/4.8.0/include/emmintrin.h:602
602       return __extension__ (__m128i)(__v16qi){
(gdb) where
#0  0x080509db in _mm_set_epi8 (__q00=1 '\001', __q01=1 '\001', 
    __q02=1 '\001', __q03=1 '\001', __q04=1 '\001', __q05=1 '\001', 
    __q06=1 '\001', __q07=1 '\001', __q08=1 '\001', __q09=1 '\001', 
    __q10=1 '\001', __q11=1 '\001', __q12=1 '\001', __q13=1 '\001', 
    __q14=1 '\001', __q15=1 '\001')
    at
/opt/tarantella/lib/gcc/i386-pc-solaris2.10/4.8.0/include/emmintrin.h:602
#1  _mm_set1_epi8 (__A=1 '\001')
    at
/opt/tarantella/lib/gcc/i386-pc-solaris2.10/4.8.0/include/emmintrin.h:637
#2  test_sse () at x.c:5
#3  0x08050a1b in main (argc=1, argv=0x8047d74) at y.c:5
(gdb) 


Example 2: libXt timer callback

Using:
- gcc 4.8.0 (stable) or gcc 4.9.0 (unstable) packages from
http://www.opencsw.org/
- or gcc 4.9.1 from tarball

File: xt.c

Result:
The stack becomes only 4-byte aligned on callback from Xt.

GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i386-pc-solaris2.10".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /export/home/justin/tmp/xt...done.
(gdb) r
Starting program: /export/home/justin/tmp/xt 

Program received signal SIGSEGV, Segmentation fault.
0x08050e97 in _mm_set_epi8 (__q00=1 '\001', __q01=1 '\001', __q02=1 '\001', 
    __q03=1 '\001', __q04=1 '\001', __q05=1 '\001', __q06=1 '\001', 
    __q07=1 '\001', __q08=1 '\001', __q09=1 '\001', __q10=1 '\001', 
    __q11=1 '\001', __q12=1 '\001', __q13=1 '\001', __q14=1 '\001', 
    __q15=1 '\001')
    at /opt/csw/lib/gcc/i386-pc-solaris2.10/4.8.0/include/emmintrin.h:602
602       return __extension__ (__m128i)(__v16qi){
(gdb) info reg ebp
ebp            0x8047c74        0x8047c74
(gdb) fr 1
#1  _mm_set1_epi8 (__A=1 '\001')
    at /opt/csw/lib/gcc/i386-pc-solaris2.10/4.8.0/include/emmintrin.h:637
637       return _mm_set_epi8 (__A, __A, __A, __A, __A, __A, __A, __A,
(gdb) info reg ebp
ebp            0x8047c74        0x8047c74
(gdb) fr 2
#2  timer_callback (client_data=0x0, id=0x8047d10) at xt.c:10
10          __m128i b = _mm_set1_epi8(1);
(gdb) info reg ebp
ebp            0x8047c74        0x8047c74
(gdb) fr 3
#3  0xfef46e29 in XtAppProcessEvent () from /usr/lib/libXt.so.4
(gdb) info reg ebp
ebp            0x8047d18        0x8047d18
(gdb) fr 4
#4  0x08050efc in main (argc=1, argv=0x8047d74) at xt.c:17
17          XtAppProcessEvent(app, XtIMTimer);
(gdb) info reg ebp
ebp            0x8047d4c        0x8047d4c
(gdb) 

I believe the problem was fixed for the limited case of Solaris 9 x86 in bug
60107. However, the analysis for this fix seems slightly off. It infers that,
because Solaris 10 creates outgoing thread stacks aligned on 16 bytes, the
32-bit ABI has changed to 16-byte stack alignment on this platform, which isn't
correct.

I suggest -mincoming-stack-boundary=2 should be the default for all 32-bit
Solaris binaries.


Workarounds:
1. __attribute__((force_align_arg_pointer)) at all entry points to gcc
generated code,
2. Compile all gcc code with -mincoming-stack-boundary=2.