[Bug tree-optimization/56252] New: Out-of-bounds array access in vectorized loop

["gael.roualland at gmail dot com" <gcc-bugzilla at gcc dot gnu dot org>]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56252

             Bug #: 56252
           Summary: Out-of-bounds array access in vectorized loop
    Classification: Unclassified
           Product: gcc
           Version: 4.4.7
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: tree-optimization
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: gael.roualland@gmail.com


Created attachment 29395
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=29395
Minimized source code exhibiting the issue

The attached minimized source generates invalid code when compiled with -O3 on
gcc-4.4 (including gcc 4.4.7), by accessing outside of an array that is besing
processed in the "foo" call.

This can be seen by running that sample code in valgrind:

$ gcc-4.4 -v
Using built-in specs.
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro
4.4.7-1ubuntu2' --with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs
--enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr
--program-suffix=-4.4 --enable-shared --enable-linker-build-id
--with-system-zlib --libexecdir=/usr/lib --without-included-gettext
--enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.4
--libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu
--enable-libstdcxx-debug --enable-objc-gc --disable-werror --with-arch-32=i686
--with-tune=generic --enable-checking=release --build=x86_64-linux-gnu
--host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.4.7 (Ubuntu/Linaro 4.4.7-1ubuntu2) 

$ gcc-4.4 -O3 vectorize-bug.c -o vectorize-bug

$ valgrind --db-attach=yes ./vectorize-bug
[...]
==21132== Invalid read of size 8
==21132==    at 0x400636: main (in /tmp/vectorize-bug/vectorize-bug)
==21132==  Address 0x51f109c is 92 bytes inside a block of size 96 alloc'd
==21132==    at 0x4C29DB4: calloc (in
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==21132==    by 0x400612: main (in /tmp/vectorize-bug/vectorize-bug)
[...]
(gdb) disassemble $rip
[...]
   0x0000000000400632 <+50>:    shufps $0x88,%xmm2,%xmm0
=> 0x0000000000400636 <+54>:    movdqu 0x10(%rdx),%xmm2
   0x000000000040063b <+59>:    paddd  %xmm0,%xmm1
[...]
(gdb) info registers 
[...]
rdx            0x51f1084        85921924

Disabling tree vectorization with -fno-tree-vectorize fixes the issue.

I understand gcc 4.4 is no longer maintained, and since gcc 4.5 and gcc 4.6 do
not exhibit the bug this has probably been fixed already, but I didn't find a
related bug, so I opted to report it anyway in case the change was a side
effect of something else.