[Bug rtl-optimization/50068] New: Invalid memory access in incr_ticks_for_insn

[uwe at netbsd dot org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=50068

             Bug #: 50068
           Summary: Invalid memory access in incr_ticks_for_insn
    Classification: Unclassified
           Product: gcc
           Version: unknown
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: rtl-optimization
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: uwe@netbsd.org


I'm not sure about a good short description for this, so please bear with me.

NetBSD is currently in a process of switching to gcc-4.5.3 as its system
compiler.  I have built a NetBSD release with the new gcc and, as a test, I
decided to build NetBSD/landisk (SuperH) on a landisk machine itself (with the
newly compiled system) to verify bootstrap/self-hosting.  While it takes
several days, it's an interesting exercise and last time I did it I actually
found a sequence point bug in out lint...  But I digressed.

The build failed with:

#   compile  libstdc++-v3/fstream-inst.o
/usr/nb/tools/bin/shle--netbsdelf-c++ -frandom-seed=4cd80fc7 -Os
-freorder-blocks -Werror -fno-implicit-templates
-fdiagnostics-show-location=once  --sysroot=/usr/nb/distrib/landisk
-I/usr/src/external/gpl3/gcc/dist/gcc -I/usr/src/external/gpl3/gcc/dist/include
-I/usr/src/external/gpl3/gcc/dist/libstdc++-v3/libsupc++
-I/usr/src/external/gpl3/gcc/lib/libstdc++-v3/../libstdc++-v3_4/arch/sh3el -I.
-DHAVE_STDLIB_H -DHAVE_STRING_H
-I/usr/src/external/gpl3/gcc/dist/libstdc++-v3/include
-I/usr/src/external/gpl3/gcc/lib/libstdc++-v3/arch/sh3el  -c
-Wno-stack-protector  -std=gnu++0x
/usr/src/external/gpl3/gcc/dist/libstdc++-v3/src/fstream-inst.cc -o
fstream-inst.o
/usr/src/external/gpl3/gcc/dist/libstdc++-v3/src/fstream-inst.cc: In member
function 'void
std::basic_fstream<char>::_ZTv0_n12_NSt13basic_fstreamIcSt11char_traitsIcEED1Ev()':
/usr/src/external/gpl3/gcc/dist/libstdc++-v3/src/fstream-inst.cc:46:1: internal
compiler error: Bus error

That SIGBUS happens in incr_ticks_for_insn at resource.c:1275 doing

  bb_ticks[b]++;

and bb_ticks is an address aligned as x*4+2.  Since it's not properly aligned
for int*, there's SIGBUS triggered.

bb_ticks is allocated at resource.c:1201 as:

  bb_ticks = XCNEWVEC (int, last_basic_block);

and for the faulty allocation the last_basic_block is 0.  Now, xcalloc in
libiberty checks that

  if (nelem == 0 || elsize == 0)
    nelem = elsize = 1;

and so it ends up with calloc(1, 1).  NetBSD's jemalloc rounds allocation
size=1 up to 2 and allocates memory at x*4+2 that IS properly aligned for
size=2, but IS NOT properly aligned for int *bb_ticks.   Since SuperH is strict
about alignment, eventual invalid access in incr_ticks_for_insn causes SIGBUS
and exposes the bug.

The compiler under investigation in the above was build on SuperH to run on
SuperH targeting SuperH.

To verify this bug I did a cross-build of NetBSD/landisk on an amd64 RedHat
machine.  Then I manually ran the command that failed in my landisk build using
valgrind.  So in this verification I used compiler built on amd64, running on
amd64 targeting SuperH.  Valgrind complained about:

==12675== Invalid read of size 4
==12675==    at 0x743398: incr_ticks_for_insn (in
netbsd/tools/libexec/gcc/shle--netbsdelf/4.5.3/cc1plus)
==12675==    by 0x741636: fill_simple_delay_slots (in
netbsd/tools/libexec/gcc/shle--netbsdelf/4.5.3/cc1plus)
==12675==    by 0x741FA9: dbr_schedule (in
netbsd/tools/libexec/gcc/shle--netbsdelf/4.5.3/cc1plus)
==12675==    by 0x8D9BC7: sh_output_mi_thunk (in
netbsd/tools/libexec/gcc/shle--netbsdelf/4.5.3/cc1plus)
==12675==    by 0x8DF98E: cgraph_expand_function (in
netbsd/tools/libexec/gcc/shle--netbsdelf/4.5.3/cc1plus)
==12675==    by 0x8E11A4: cgraph_optimize (in
netbsd/tools/libexec/gcc/shle--netbsdelf/4.5.3/cc1plus)
==12675==    by 0x8E1A54: cgraph_finalize_compilation_unit (in
netbsd/tools/libexec/gcc/shle--netbsdelf/4.5.3/cc1plus)
==12675==    by 0x4D2BF4: cp_write_global_declarations (in
netbsd/tools/libexec/gcc/shle--netbsdelf/4.5.3/cc1plus)
==12675==    by 0x78D905: do_compile (in
netbsd/tools/libexec/gcc/shle--netbsdelf/4.5.3/cc1plus)
==12675==    by 0x78E04D: toplev_main (in
netbsd/tools/libexec/gcc/shle--netbsdelf/4.5.3/cc1plus)
==12675==    by 0x35C801D973: (below main) (in /lib64/libc-2.5.so)
==12675==  Address 0x4f22524 is 3 bytes after a block of size 1 alloc'd
==12675==    at 0x4A0516B: calloc (vg_replace_malloc.c:418)
==12675==    by 0xA2C628: xcalloc (in
netbsd/tools/libexec/gcc/shle--netbsdelf/4.5.3/cc1plus)
==12675==    by 0x74412B: init_resource_info (in
netbsd/tools/libexec/gcc/shle--netbsdelf/4.5.3/cc1plus)
==12675==    by 0x741F62: dbr_schedule (in
netbsd/tools/libexec/gcc/shle--netbsdelf/4.5.3/cc1plus)
==12675==    by 0x8D9BC7: sh_output_mi_thunk (in
netbsd/tools/libexec/gcc/shle--netbsdelf/4.5.3/cc1plus)
==12675==    by 0x8DF98E: cgraph_expand_function (in
netbsd/tools/libexec/gcc/shle--netbsdelf/4.5.3a/cc1plus)
==12675==    by 0x8E11A4: cgraph_optimize (in
netbsd/tools/libexec/gcc/shle--netbsdelf/4.5.3/cc1plus)
==12675==    by 0x8E1A54: cgraph_finalize_compilation_unit (in
netbsd/tools/libexec/gcc/shle--netbsdelf/4.5.3/cc1plus)
==12675==    by 0x4D2BF4: cp_write_global_declarations (in
netbsd/tools/libexec/gcc/shle--netbsdelf/4.5.3/cc1plus)
==12675==    by 0x78D905: do_compile (in
netbsd/tools/libexec/gcc/shle--netbsdelf/4.5.3/cc1plus)
==12675==    by 0x78E04D: toplev_main (in
netbsd/tools/libexec/gcc/shle--netbsdelf/4.5.3/cc1plus)
==12675==    by 0x35C801D973: (below main) (in /lib64/libc-2.5.so)

thus confirming the bug.