Bug 36968 - [4.4 regression] malloc corruption building jv-convert.exe
Summary: [4.4 regression] malloc corruption building jv-convert.exe
Alias: None
Product: gcc
Classification: Unclassified
Component: other (show other bugs)
Version: 4.4.0
: P3 normal
Target Milestone: 4.4.0
Assignee: Aaron W. LaFramboise
Keywords: build, patch
Depends on:
Blocks: 36216
  Show dependency treegraph
Reported: 2008-07-30 02:17 UTC by Aaron W. LaFramboise
Modified: 2008-08-11 00:22 UTC (History)
1 user (show)

See Also:
Host: i386-pc-mingw32
Known to work: 4.3.1
Known to fail: 4.4.0
Last reconfirmed: 2008-07-30 02:17:59

Allocate argv array first (1.24 KB, patch)
2008-07-31 22:59 UTC, Aaron W. LaFramboise
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Aaron W. LaFramboise 2008-07-30 02:17:30 UTC
When bootstrapping:

/bin/sh ./libtool --tag=GCJ --mode=link /mingw/src/gccf/gcc/gcj -B/mingw/src/gcc
f/i386-pc-mingw32/libjava/ -B/mingw/src/gccf/gcc/ -L/mingw/src/gccf/i386-pc-ming
w32/libjava -ffloat-store -fomit-frame-pointer -Usun -fno-omit-frame-pointer -g 
-O2  -o jv-convert.exe --main=gnu.gcj.convert.Convert -rpath /mingw/lib/gcc/i386
-pc-mingw32/4.4.0 -shared-libgcc    -L/mingw/src/gccf/i386-pc-mingw32/libjava/.l
ibs libgcj.la 
libtool: link: /mingw/src/gccf/gcc/gcj -B/mingw/src/gccf/i386-pc-mingw32/libjava
/ -B/mingw/src/gccf/gcc/ -ffloat-store -fomit-frame-pointer -Usun -fno-omit-fram
e-pointer -g -O2 -o .libs/jv-convert.exe --main=gnu.gcj.convert.Convert -shared-
libgcc  -L/mingw/src/gccf/i386-pc-mingw32/libjava/.libs -L/mingw/src/gccf/i386-p
c-mingw32/libjava ./.libs/libgcj.a -L/mingw/lib/gcc/i386-pc-mingw32/4.4.0

gcj.exe: out of memory allocating 160 bytes
make[3]: *** [jv-convert.exe] Error 1

With mpatrol, we can isolate the actual bug:

MPATROL_OPTIONS='PAGEALLOC=LOWER' gdb /mingw/src/gccf/gcc/gcj

(gdb) set args -B/mingw/src/gccf/i386-pc-mingw32/libjava/ -B/mingw/src/gccf/gcc/ -ffloat-store -fomit-frame-pointer -Usun -fno-omit-frame-pointer -g -O2 -o .libs/jv-convert.exe --main=gnu.gcj.convert.Convert -shared-libgcc  -L/mingw/src/gccf/i386-pc-mingw32/libjava/.libs -L/mingw/src/gccf/i386-pc-mingw32/libjava ./.libs/libgcj.a -L/mingw/lib/gcc/i386-pc-mingw32/4.4.0

(gdb) run
Program received signal SIGSEGV, Segmentation fault.                        
0x00414ed8 in spawn_script (executable=0x36a7000 "/mingw/src/gccf/gcc/as",  
    argv=0x12a3ffc, env=0x0, dwCreationFlags=0, si=0x27fb60, pi=0x27fb50)   
    at ../../svn/libiberty/pex-win32.c:654                                  
654                   *avhere = executable1;                                

(gdb) bt
#0  0x00414ed8 in spawn_script (                                                
    executable=0x36a7000 "/mingw/src/gccf/gcc/as", argv=0x12a3ffc, env=0x0,     
    dwCreationFlags=0, si=0x27fb60, pi=0x27fb50)                                
    at ../../svn/libiberty/pex-win32.c:654                                      
#1  0x00415113 in pex_win32_exec_child (obj=0x36aa000, flags=1,                 
    executable=0x36a7000 "/mingw/src/gccf/gcc/as", argv=0x12a4000, env=0x0,     
    in=0, out=1, errdes=2, toclose=-1, errmsg=0x27fcc8, err=0x27fdb4)           
    at ../../svn/libiberty/pex-win32.c:784                                      
#2  0x0041cc55 in pex_run_in_environment (obj=0x36aa000,                        
    flags=<value optimized out>,                                                
    executable=0x36a7000 "/mingw/src/gccf/gcc/as", argv=0x12a4000, env=0x0,     
    orig_outname=0x0, errname=0x0, err=0x27fdb4)                                
    at ../../svn/libiberty/pex-common.c:342                                     
#3  0x0041ce3f in pex_run (obj=0x36aa000, flags=1,                              
    executable=0x36a7000 "/mingw/src/gccf/gcc/as", argv=0x12a4000,              
    orig_outname=0x0, errname=0x0, err=0x27fdb4)                                
    at ../../svn/libiberty/pex-common.c:372                                     
#4  0x004040cb in execute () at ../../svn/gcc/gcc.c:3005                        
#5  0x0040d0be in lang_specific_pre_link () at ../../svn/gcc/java/jvspec.c:673  
#6  0x0040c42c in main (argc=1852400220, argv=0x68735c)                         
    at ../../svn/gcc/gcc.c:6825                                                 

The problem is clearly on pex-win32.c:646:

const char ** avhere = (const char **) --argv;

Then we hit the fault a few statements down at 654:

*avhere = executable1;

This is writing at (argv-1) which overrides the heap block header and causes the corruption.

I'm going to look at this more tomorrow and see if I can figure out why its doing this.  I'm a little curious how this code has been like this since 2005 without ever causing trouble before.
Comment 1 Aaron W. LaFramboise 2008-07-31 22:59:26 UTC
Created attachment 15987 [details]
Allocate argv array first
Comment 2 Aaron W. LaFramboise 2008-08-11 00:22:34 UTC