while testing ffmpeg compiled with gcc 4.3.3 from 4.3 trunk I got a lot of crashes especially in snow encoding testcase by using: -fno-tree-dominator-opts -fno-tree-vrp -fno-dce -fno-tree-ch I get no crashes gcc 4.4 has same problems, I've already filled a bug about tree-ch, that corrupts the stack on win32 (looks like it's related to alloca())
No other target has this issue except Win32. I am going to say mingw support for alloca is broken somehow.
this problem started with 4.3, 4.3.2 on debian linux hasn't these problems
As usual we would need preprocessed source as a testcase.
unfortunately snow.c is a very big file, I'll try to find a shorter example, but if the problem is in alloca() the generated asm will not be so usefull
*** Bug 37584 has been marked as a duplicate of this bug. ***
first bt, (pls tell me if you need output of leave temps, generated asm preprocessed or other stuff) Program received signal SIGSEGV, Segmentation fault. 0x0092c4c2 in _alloca () (gdb) bt #0 0x0092c4c2 in _alloca () #1 0x005c7462 in get_dc (s=0x6230030, mb_x=0, mb_y=0, plane_index=0) at libavcodec/snow.c:2426 #2 0x005cff6a in iterative_me (s=0x6230030) at libavcodec/snow.c:3126 #3 0x005d7467 in encode_blocks (s=0x6230030, search=1) at libavcodec/snow.c:2074 #4 0x005d7a2a in encode_frame (avctx=0x5aa3f90, buf=0x5fe06c0 "<BINARY DATA REMOVED>"..., buf_size=262144, data=0x22eda8) at libavcodec/snow.c:4268 #5 0x00499513 in avcodec_encode_video (avctx=0x5aa3f90, buf=0x5fe06c0 "<BINARY DATA REMOVED>"..., buf_size=262144, pict=0x22eda8) at libavcodec/utils.c:894 #6 0x00405626 in output_packet (ist=0x5ed6450, ist_index=0, ost_table=0x5aa4390, nb_ostreams=1, pkt=0x22fed8) at ffmpeg.c:954 #7 0x00409269 in av_encode (output_files=0xb6b378, nb_output_files=1, input_files=0xb6a608, nb_input_files=1, stream_maps=0xb6b3c8, nb_stream_maps=0) at ffmpeg.c:2116 #8 0x00409761 in main (argc=Cannot access memory at address 0x0 ) at ffmpeg.c:3888
compile flags OPTFLAGS=-O3 -fno-common -g3 -march=native -mtune=native -pipe -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_ISOC99_SOURCE -D_POSIX_C_SOURCE=200112 -fasm -std=c99 -fomit-frame-pointer -DPTW32_STATIC_LIB -g3 -Wdeclaration-after-statement -Wall -Wno-switch -Wdisabled-optimization -Wpointer-arith -Wredundant-decls -Wno-pointer-sign -Wcast-qual -Wwrite-strings -Wtype-limits -O2 -fno-math-errno -fno-signed-zeros -O3 -fno-common -march=native -mtune=native -pipe where added by me (-fno-common is needed to avoid crash in sse code)
another crash in snow, this time argc/argv is not screwed Program received signal SIGSEGV, Segmentation fault. 0x0092c4c2 in _alloca () (gdb) bt #0 0x0092c4c2 in _alloca () #1 0x005d7de7 in encode_frame (avctx=0x5aa3f20, buf=0x5fda480 "<BINARY DATA REMOVED>"..., buf_size=405504, data=0x22eda8) at libavcodec/snow.c:2426 #2 0x00499513 in avcodec_encode_video (avctx=0x5aa3f20, buf=0x5fda480 "<BINARY DATA REMOVED>"..., buf_size=405504, pict=0x22eda8) at libavcodec/utils.c:894 #3 0x00405626 in output_packet (ist=0x5ed6450, ist_index=0, ost_table=0x5cd6470, nb_ostreams=1, pkt=0x22fed8) at ffmpeg.c:954 #4 0x00409269 in av_encode (output_files=0xb6b378, nb_output_files=1, input_files=0xb6a608, nb_input_files=1, stream_maps=0xb6b3c8, nb_stream_maps=0) at ffmpeg.c:2116 #5 0x00409761 in main (argc=95043360, argv=0x3f4998) at ffmpeg.c:3888
sqv1 Program received signal SIGSEGV, Segmentation fault. 0x0092c4c2 in _alloca () (gdb) bt #0 0x0092c4c2 in _alloca () #1 0x005e3338 in svq1_encode_plane (s=0x5a9f2c0, plane=<value optimized out>, src_plane=0x5ac9fa0 "<BINARY DATA REMOVED>"..., ref_plane=0x6063010 '\200' <repeats 200 times>..., decoded_plane=0x6089300 '\200' <repeats 200 times>..., width=352, height=288, src_stride=352, stride=416) at libavcodec/svq1enc.c:363 #2 0x005e467a in svq1_encode_frame (avctx=0x5aa3eb0, buf=0x5fe05a0 "", buf_size=405504, data=0x22eda8) at libavcodec/svq1enc.c:541 #3 0x00499513 in avcodec_encode_video (avctx=0x5aa3eb0, buf=0x5fe05a0 "", buf_size=405504, pict=0x22eda8) at libavcodec/utils.c:894 #4 0x00405626 in output_packet (ist=0x5ed6450, ist_index=0, ost_table=0x5ac9e00, nb_ostreams=1, pkt=0x22fed8) at ffmpeg.c:954 #5 0x00409269 in av_encode (output_files=0xb6b378, nb_output_files=1, input_files=0xb6a608, nb_input_files=1, stream_maps=0xb6b3c8, nb_stream_maps=0) at ffmpeg.c:2116 #6 0x00409761 in main (argc=Cannot access memory at address 0x1000000 ) at ffmpeg.c:3888
Does this issue appears also, when using builtin alloca version? As I noticed does the switch -fno-builtin shows explict broken _alloca for x64. The call-save area isn't adjusted and compiler seems not to take care here
I'm using 4.5 from svn, with -O2 and looks like not affected 4.3 and 4.4 are almost unusable on mingw (at least my builds) something changed in 4.5 branch, I've not tested further 4.3 or .4.4 since I was using 4.2 then switched to 4.5 when it started to work on mingw
however 4.5 is still far from being stable as 4.2, I get many crashes while using complied mplayer (it's a stress test for gcc :)) Unfortunately I had no much time to debug mplayer builds
I've just run into this problem too with MinGW's packaged GCC 4.4.0. It was working fine for some time but recently started crashing when both libevent and the C++ program that libevent was calling back into were both compiled at at least -O1. This causes libevent to use the edi register whose value is corrupted (set to null, actually) by a callback into our C++ program. The C++ app seems to use the edi register at any optimisation level and in any case seems to push it onto the stack and pop it off at the end of the callback, but some kind of stack corruption goes on within the callback that means the value popped off the stack at the end is zero when the app is compiled at -O1. Compiling the app at -O0 or at -O1 with -fno-dce prevents the crash, although compiling with -O2 -fno-dce causes the same crash.
alloca has been fixed already so closing as fixed.