[Bug c/67442] New: GCC 5.2.0 on x86_64 creates invalid address on specific array index calculation through pointer

["jussi.judin at ericsson dot com" <gcc-bugzilla at gcc dot gnu dot org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67442

            Bug ID: 67442
           Summary: GCC 5.2.0 on x86_64 creates invalid address on
                    specific array index calculation through pointer
           Product: gcc
           Version: unknown
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c
          Assignee: unassigned at gcc dot gnu.org
          Reporter: jussi.judin at ericsson dot com
  Target Milestone: ---

GCC 5.2.0 and the current git version
(ae436f3bdc66153005b7a260e0239521679f3965) create invalid address to access
array index calculated in specific way on x86_64 platform:

short foo[100];

int main()
{
    short* bar = &foo[50];
    short i = 1;
    short j = 1;
    short value = bar[8 - i * 2 * j];
    return value;
}

$ ~/local/gcc-bin/bin/gcc -v -save-temps -Wall -Wextra -g3 -ggdb tst.c
$ ./a.out
Segmentation fault (core dumped)
$ gdb a.out core
Core was generated by `./a.out'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00000000004004d5 in main () at tst.c:8
8           short value = bar[8 - i * 2 * j];
(gdb) disassemble /m 0x00000000004004d5
8           short value = bar[8 - i * 2 * j];
   0x00000000004004ae <+24>:    movswq -0xa(%rbp),%rdx
   0x00000000004004b3 <+29>:    movswq -0xc(%rbp),%rax
   0x00000000004004b8 <+34>:    imul   %rax,%rdx
   0x00000000004004bc <+38>:    mov    %rdx,%rax
   0x00000000004004bf <+41>:    shl    $0x1e,%rax
   0x00000000004004c3 <+45>:    sub    %rdx,%rax
   0x00000000004004c6 <+48>:    shl    $0x2,%rax
   0x00000000004004ca <+52>:    lea    0x10(%rax),%rdx
   0x00000000004004ce <+56>:    mov    -0x8(%rbp),%rax
   0x00000000004004d2 <+60>:    add    %rdx,%rax
=> 0x00000000004004d5 <+63>:    movzwl (%rax),%eax
   0x00000000004004d8 <+66>:    mov    %ax,-0xe(%rbp)
(gdb) p &bar[8 - i * 2 * j]
$3 = (short *) 0x600970 <foo+112>
(gdb) p/x $rax
$2 = 0x100600970

So there is 0x100000000 added somewhere to the address that is used for the
actual array access. I have seen value 0x500000000 added in the production code
I encountered this issue. It seems to need following parts to trigger:
reference to an array (&foo[50], foo could also be on stack, but it creates
more nasty addresses), array index access through pointer by [const1 - var1 *
const2 * var2], where const1 > 0 and const2 > 1. I just used values 8 and 2 to
demonstrate that we don't access negative indexes, whereas the actual signal
processing code I encountered this does.

If I assign this array index to a variable and use it to access the array
through pointer, then GCC creates correct binary. Also if I use -m32 to compile
for x86, this does not result in segmentation fault. So it looks like it's
x86_64 specific issue.

The system I used to compile this GCC is x86_64 Ubuntu 14.04 based one. GCC
returns following information about itself:

$ ~/local/gcc-bin/bin/gcc -v
Using built-in specs.
COLLECT_GCC=/home/ejusjud/local/gcc-bin/bin/gcc
COLLECT_LTO_WRAPPER=/local/ejusjud/gcc-bin/bin/../libexec/gcc/x86_64-pc-linux-gnu/6.0.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: ../gcc/configure --prefix=/home/ejusjud/local/gcc-bin
Thread model: posix
gcc version 6.0.0 20150902 (experimental) (GCC) 


And about the tools and other libraries during the compilation:


GNU C11 (GCC) version 6.0.0 20150902 (experimental) (x86_64-pc-linux-gnu)
        compiled by GNU C version 6.0.0 20150902 (experimental), GMP version
5.1.3, MPFR version 3.1.2-p3, MPC version 1.0.1
GGC heuristics: --param ggc-min-expand=30 --param ggc-min-heapsize=4096
GNU C11 (GCC) version 6.0.0 20150902 (experimental) (x86_64-pc-linux-gnu)
        compiled by GNU C version 6.0.0 20150902 (experimental), GMP version
5.1.3, MPFR version 3.1.2-p3, MPC version 1.0.1
GGC heuristics: --param ggc-min-expand=30 --param ggc-min-heapsize=4096
Compiler executable checksum: 2b4ce747c000701038232915f15d53af
COLLECT_GCC_OPTIONS='-v' '-save-temps' '-Wall' '-Wextra' '-g3' '-ggdb'
'-mtune=generic' '-march=x86-64'
 as -v --64 -o tst.o tst.s
GNU assembler version 2.24 (x86_64-linux-gnu) using BFD version (GNU Binutils
for Ubuntu) 2.24