| Summary: | ffmpeg runtime segfault with lto | ||
|---|---|---|---|
| Product: | gcc | Reporter: | Manuel Lauss <manuel.lauss> |
| Component: | lto | Assignee: | Martin Liška <marxin> |
| Status: | RESOLVED INVALID | ||
| Severity: | normal | CC: | cehoyos |
| Priority: | P3 | ||
| Version: | 8.0.1 | ||
| Target Milestone: | --- | ||
| Host: | Target: | ||
| Build: | Known to work: | ||
| Known to fail: | Last reconfirmed: | 2018-03-31 00:00:00 | |
Hi Manual. Can you please reduce a bit the configure options. The package requires many libraries as dependency. How do you invoke ffmpeg to expose the crash? This is the most minimal ffmpeg config that can trigger the segfault. The interesting switch is "--enabled-hardcoded-tables". I use the "mpv" Player with a standard mp4 video file to trigger it (https://github.com/mpv-player/mpv/archive/v0.27.2.zip) configure --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --docdir=/usr/share/doc/ffmpeg-3.4.2-r1/html --mandir=/usr/share/man --enable-share d --cc=x86_64-pc-linux-gnu-gcc --cxx=x86_64-pc-linux-gnu-g++ --ar=x86_64-pc-linux-gnu-ar --optflags=-O3 -ggdb -march=znver1 -mtune=znver1 -flto=16 -fno-fat-lto-objects -pipe --disable-static --enable-avfilter --enable-avresample --disable-stripping --disable-encoders --disable-indev=v4l2 --disable-outdev=v4l2 --disable-indev=alsa --disable-indev=oss --disable -indev=jack --disable-outdev=alsa --disable-outdev=oss --disable-outdev=sdl --disable-bzlib --disable-runtime-cpudetect --disable-debug --disable-gcrypt --disable-gnutls --disable-gmp --enable-gpl --enable-hardco ded-tables --disable-iconv --disable-lzma --disable-network --disable-opencl --disable-openssl --disable-postproc --disable-libsmbclient --disable-ffplay --disable-sdl2 --disable-vaapi --disable-vdpau --disable-x lib --disable-libxcb --disable-libxcb-shm --disable-libxcb-xfixes --disable-zlib --disable-libcdio --disable-libiec61883 --disable-libdc1394 --disable-libcaca --disable-openal --disable-opengl --disable-libv4l2 - -disable-libpulse --disable-libdrm --disable-libopencore-amrwb --disable-libopencore-amrnb --disable-libfdk-aac --disable-libopenjpeg --disable-libbluray --disable-libcelt --disable-libgme --disable-libgsm --disa ble-mmal --disable-libmodplug --disable-libopus --disable-libilbc --disable-librtmp --disable-libssh --disable-libspeex --disable-librsvg --disable-libvorbis --disable-libvpx --disable-libzvbi --disable-appkit -- disable-libbs2b --disable-chromaprint --disable-libflite --disable-frei0r --disable-libfribidi --disable-fontconfig --disable-ladspa --disable-libass --disable-libfreetype --disable-librubberband --disable-libzmq --disable-libzimg --disable-libsoxr --enable-pthreads --disable-armv5te --disable-armv6 --disable-armv6t2 --disable-neon --disable-vfp --disable-vfpv3 --disable-armv8 --disable-mipsdsp --disable-mipsdspr2 --disa ble-mipsfpu --disable-altivec --disable-amd3dnow --disable-amd3dnowext --disable-fma4 --disable-xop --cpu=znver1 --enable-lto --disable-doc --disable-htmlpages --enable-manpages (In reply to Manuel Lauss from comment #2) > This is the most minimal ffmpeg config that can trigger the segfault. The > interesting switch is "--enabled-hardcoded-tables". I use the "mpv" Player > with a standard mp4 video file to trigger it > (https://github.com/mpv-player/mpv/archive/v0.27.2.zip) Good. so we have minimal set of options for ffmpeg. Now please explain me how to build the MPV tool? Is it really needed? Can't one trigger that directly with ffmepg? Please upload somewhere the mp4 file so that I can reproduce it. (In reply to Martin Liška from comment #3) > (In reply to Manuel Lauss from comment #2) > > This is the most minimal ffmpeg config that can trigger the segfault. The > > interesting switch is "--enabled-hardcoded-tables". I use the "mpv" Player > > with a standard mp4 video file to trigger it > > (https://github.com/mpv-player/mpv/archive/v0.27.2.zip) > > Good. so we have minimal set of options for ffmpeg. Now please explain me > how to build the MPV tool? Is it really needed? Can't one trigger that > directly with ffmepg? Please upload somewhere the mp4 file so that I can > reproduce it. The segfault happens during initialization of ffmpeg when mpv player starts. I can also trigger it with firefox on e.g. youtube: it crashes the tab. ad mpv: unpack, configure with the following line (you need python3): CCFLAGS="-O3 -march=znver1 -mtune=znver1 -flto=16 -fno-fat-lto-objects -pipe" LINKFLAGS="-O3 -march=znver1 -mtune=znver1 -flto=16 -fno-fat-lto-objects -pipe -Wl,--no-as-needed" ./waf --prefix=/usr --libdir=/usr/lib64 --confdir=/etc/mpv --docdir=/usr/share/doc/mpv-0.27.2 --htmldir=/usr/share/doc/mpv-0.27.2/html --enable-libmpv-shared --disable-libmpv-static --disable-static-build --disable-optimize --disable-debug-build --enable-html-build --disable-pdf-build --enable-cplugins --disable-zsh-comp --disable-test --disable-android --enable-iconv --disable-libsmbclient --enable-lua --lua=luajit --disable-javascript --enable-x11 --disable-build-date configure ./waf build (In reply to Manuel Lauss from comment #4) > (In reply to Martin Liška from comment #3) > > (In reply to Manuel Lauss from comment #2) > > > This is the most minimal ffmpeg config that can trigger the segfault. The > > > interesting switch is "--enabled-hardcoded-tables". I use the "mpv" Player > > > with a standard mp4 video file to trigger it > > > (https://github.com/mpv-player/mpv/archive/v0.27.2.zip) > > > > Good. so we have minimal set of options for ffmpeg. Now please explain me > > how to build the MPV tool? Is it really needed? Can't one trigger that > > directly with ffmepg? Please upload somewhere the mp4 file so that I can > > reproduce it. > > The segfault happens during initialization of ffmpeg when mpv player starts. > I can also trigger it with firefox on e.g. youtube: it crashes the tab. Ok, so please download following sample video: http://techslides.com/demos/sample-videos/small.mp4 And try to run ffmpeg to print info: ./ffmpeg -i small.mp4 Does it also segfault? > > > ad mpv: unpack, configure with the following line (you need python3): > CCFLAGS="-O3 -march=znver1 -mtune=znver1 -flto=16 -fno-fat-lto-objects > -pipe" LINKFLAGS="-O3 -march=znver1 -mtune=znver1 -flto=16 > -fno-fat-lto-objects -pipe -Wl,--no-as-needed" ./waf --prefix=/usr > --libdir=/usr/lib64 --confdir=/etc/mpv --docdir=/usr/share/doc/mpv-0.27.2 > --htmldir=/usr/share/doc/mpv-0.27.2/html --enable-libmpv-shared > --disable-libmpv-static --disable-static-build --disable-optimize > --disable-debug-build --enable-html-build --disable-pdf-build > --enable-cplugins --disable-zsh-comp --disable-test --disable-android > --enable-iconv --disable-libsmbclient --enable-lua --lua=luajit > --disable-javascript --enable-x11 --disable-build-date configure > > ./waf build How the player knows where to find the built ffmpeg libraries? (In reply to Martin Liška from comment #5) > (In reply to Manuel Lauss from comment #4) > > (In reply to Martin Liška from comment #3) > > > (In reply to Manuel Lauss from comment #2) > > > > This is the most minimal ffmpeg config that can trigger the segfault. The > > > > interesting switch is "--enabled-hardcoded-tables". I use the "mpv" Player > > > > with a standard mp4 video file to trigger it > > > > (https://github.com/mpv-player/mpv/archive/v0.27.2.zip) > > > > > > Good. so we have minimal set of options for ffmpeg. Now please explain me > > > how to build the MPV tool? Is it really needed? Can't one trigger that > > > directly with ffmepg? Please upload somewhere the mp4 file so that I can > > > reproduce it. > > > > The segfault happens during initialization of ffmpeg when mpv player starts. > > I can also trigger it with firefox on e.g. youtube: it crashes the tab. > > Ok, so please download following sample video: > http://techslides.com/demos/sample-videos/small.mp4 > > And try to run ffmpeg to print info: > ./ffmpeg -i small.mp4 > > Does it also segfault? Yes, same backtrace, same fault location. (In reply to Manuel Lauss from comment #6) > (In reply to Martin Liška from comment #5) > > (In reply to Manuel Lauss from comment #4) > > > (In reply to Martin Liška from comment #3) > > > > (In reply to Manuel Lauss from comment #2) > > > > > This is the most minimal ffmpeg config that can trigger the segfault. The > > > > > interesting switch is "--enabled-hardcoded-tables". I use the "mpv" Player > > > > > with a standard mp4 video file to trigger it > > > > > (https://github.com/mpv-player/mpv/archive/v0.27.2.zip) > > > > > > > > Good. so we have minimal set of options for ffmpeg. Now please explain me > > > > how to build the MPV tool? Is it really needed? Can't one trigger that > > > > directly with ffmepg? Please upload somewhere the mp4 file so that I can > > > > reproduce it. > > > > > > The segfault happens during initialization of ffmpeg when mpv player starts. > > > I can also trigger it with firefox on e.g. youtube: it crashes the tab. > > > > Ok, so please download following sample video: > > http://techslides.com/demos/sample-videos/small.mp4 > > > > And try to run ffmpeg to print info: > > ./ffmpeg -i small.mp4 > > > > Does it also segfault? > > Yes, same backtrace, same fault location. Hm, I put __builtin_abort to: static av_cold void aac_static_table_init(void) function. And it's unfortunately called. Probably because: $ ./ffmpeg -i small.mp4 test.mp3 ... Decoder (codec aac) not found for input stream #0:1 Are you sure the configure options enable the codec (AAC)? Stupid mistake from me. Now I can reproduce that: LD_LIBRARY_PATH=./libavcodec/ gdb --args ./ffmpeg -i small.mp4 So it's really caused by --enable-hardcoded-tables. With that you have in sinewin.c pre-processed source file: extern const INTFLOAT __attribute__ ((aligned (32))) ff_sine_960[960]; As you can se, it's const (a.k.a. READONLY). Thus doing a write to it in ff_sine_window_init causes a segfault. Thus closing as invalid. Built with gcc-7.3.0 and LTO it works fine; is this a bug in 7.3.0 instead then? It was changed on trunk in:
commit 371072bf395be11f36ef31bb3cfec06bbfc58597
Author: law <law@138bc75d-0d04-0410-961f-82ee72b054a4>
Date: Fri Sep 1 16:26:00 2017 +0000
* varasm.c (bss_initializer_p): Do not put constants into .bss
(categorize_decl_for_section): Handle bss_initializer_p returning
false when DECL_INITIAL is NULL.
* gcc.target/i386/const-in-bss.c: New test.
git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@251602 138bc75d-0d04-0410-961f-82ee72b054a4
Note the using an undefined behavior gives optimizer opportunity to optimize to any way. In this case GCC 7 generated working and GCC 8 not working code in case of ffmpeg.
Should be fixed: http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=10f46815 https://trac.ffmpeg.org/ticket/7491 I wonder a little how the FFmpeg developers were supposed to find out about this bug... (In reply to Carl Eugen Hoyos from comment #12) > Should be fixed: > http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=10f46815 > https://trac.ffmpeg.org/ticket/7491 > > I wonder a little how the FFmpeg developers were supposed to find out about > this bug... Good, thanks for mainline fix! |
ffmpeg-3.4.2 built with gcc-8 as of 20180330 and LTO enabled causes a runtime segfault: Thread 1 "mpv" received signal SIGSEGV, Segmentation fault. 0x00007ffff686b305 in ff_sine_window_init (window=0x7ffff739a2c0 <ff_sine_960>, n=960) at src/libavcodec/sinewin_tablegen.h:73 73 window[i] = SIN_FIX(sinf((i + 0.5) * (M_PI / (2.0 * n)))); (gdb) bt #0 0x00007ffff686b305 in ff_sine_window_init (window=0x7ffff739a2c0 <ff_sine_960>, n=960) at src/libavcodec/sinewin_tablegen.h:73 #1 0x00007ffff6861d54 in aac_static_table_init () at src/libavcodec/aacdec_template.c:1142 #2 0x00007ffff511d187 in __pthread_once_slow (once_control=0x7ffff7b4cf20 <aac_table_init.lto_priv>, init_routine=0x7ffff6861256 <aac_static_table_init>) at pthread_once.c:116 #3 0x00007ffff6862aae in aac_decode_init (avctx=0xa5ef00) at src/libavcodec/aacdec_template.c:1159 #4 0x00007ffff6ecbef6 in avcodec_open2 (avctx=0xa5ef00, codec=<optimized out>, options=<optimized out>) at src/libavcodec/utils.c:1020 #5 0x0000000000502b3a in ?? () #6 0x00000000004fb7be in ?? () #7 0x000000000045afb7 in ?? () #8 0x000000000045b8af in ?? () #9 0x00000000004d3246 in ?? () #10 0x000000000040fa21 in ?? () #11 0x00007ffff4c8854b in __libc_start_main (main=0x40f9d0, argc=2, argv=0x7fffffffd1c8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffd1b8) at ../csu/libc-start.c:308 #12 0x000000000040fb9a in ?? () (gdb) disass Dump of assembler code for function ff_sine_window_init: 0x00007ffff686b2c9 <+0>: push %r12 0x00007ffff686b2cb <+2>: mov %rdi,%r12 0x00007ffff686b2ce <+5>: push %rbp 0x00007ffff686b2cf <+6>: mov %esi,%ebp 0x00007ffff686b2d1 <+8>: push %rbx 0x00007ffff686b2d2 <+9>: xor %ebx,%ebx 0x00007ffff686b2d4 <+11>: cmp %ebx,%ebp 0x00007ffff686b2d6 <+13>: jle 0x7ffff686b310 <ff_sine_window_init+71> 0x00007ffff686b2d8 <+15>: vmovsd 0x878108(%rip),%xmm2 # 0x7ffff70e33e8 0x00007ffff686b2e0 <+23>: vcvtsi2sd %ebp,%xmm0,%xmm0 0x00007ffff686b2e4 <+27>: vaddsd %xmm0,%xmm0,%xmm0 0x00007ffff686b2e8 <+31>: vcvtsi2sd %ebx,%xmm1,%xmm1 0x00007ffff686b2ec <+35>: vaddsd 0x87811c(%rip),%xmm1,%xmm1 # 0x7ffff70e3410 0x00007ffff686b2f4 <+43>: vdivsd %xmm0,%xmm2,%xmm0 0x00007ffff686b2f8 <+47>: vmulsd %xmm1,%xmm0,%xmm0 0x00007ffff686b2fc <+51>: vcvtsd2ss %xmm0,%xmm0,%xmm0 0x00007ffff686b300 <+55>: callq 0x7ffff6801db0 <sinf@plt> => 0x00007ffff686b305 <+60>: vmovss %xmm0,(%r12,%rbx,4) 0x00007ffff686b30b <+66>: inc %rbx 0x00007ffff686b30e <+69>: jmp 0x7ffff686b2d4 <ff_sine_window_init+11> 0x00007ffff686b310 <+71>: pop %rbx 0x00007ffff686b311 <+72>: pop %rbp 0x00007ffff686b312 <+73>: pop %r12 0x00007ffff686b314 <+75>: retq End of assembler dump. (gdb) info registers rax 0x0 0 rbx 0x0 0 rcx 0x1 1 rdx 0x78 120 rsi 0x3c0 960 rdi 0x7ffff739a2c0 140737341137600 rbp 0x3c0 0x3c0 rsp 0x7fffffffc840 0x7fffffffc840 r8 0x7fffffffa850 140737488332880 r9 0x7 7 r10 0xfffffffffffffd07 -761 r11 0x7ffff5682c90 140737310633104 r12 0x7ffff739a2c0 140737341137600 r13 0x7fffffffc9f8 140737488341496 r14 0x0 0 r15 0x0 0 rip 0x7ffff686b305 0x7ffff686b305 <ff_sine_window_init+60> eflags 0x10202 [ IF RF ] cs 0x33 51 ss 0x2b 43 ds 0x0 0 es 0x0 0 fs 0x0 0 gs 0x0 0 Without LTO it's fine, the assembly of the faulting function is the same. I tried to isolate a testcase but failed. ffmpeg built with: configure --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --docdir=/usr/share/doc/ffmpeg-3.4.2-r1/html --mandir=/usr/share/man --enable-share d --cc=x86_64-pc-linux-gnu-gcc --cxx=x86_64-pc-linux-gnu-g++ --ar=x86_64-pc-linux-gnu-ar --optflags=-O3 -ggdb -march=znver1 -mtune=znver1 -flto=16 -fno-fat-lto-objects -fno-strict-aliasing -fexpensive-optimizatio ns -Wno-deprecated -pipe --disable-static --enable-avfilter --enable-avresample --disable-stripping --enable-nonfree --enable-version3 --disable-indev=alsa --disable-indev=oss --disable-indev=jack --disable-outde v=alsa --disable-outdev=oss --enable-version3 --enable-version3 --enable-bzlib --disable-runtime-cpudetect --disable-debug --disable-gcrypt --enable-gnutls --enable-gmp --enable-gpl --enable-hardcoded-tables --en able-iconv --enable-lzma --enable-network --disable-opencl --enable-openssl --enable-postproc --disable-libsmbclient --enable-ffplay --enable-sdl2 --disable-vaapi --enable-vdpau --enable-xlib --enable-libxcb --en able-libxcb-shm --enable-libxcb-xfixes --enable-zlib --disable-libcdio --disable-libiec61883 --disable-libdc1394 --disable-libcaca --enable-openal --enable-opengl --enable-libv4l2 --enable-libpulse --disable-libd rm --enable-libopencore-amrwb --enable-libopencore-amrnb --disable-libfdk-aac --enable-libopenjpeg --enable-libbluray --disable-libcelt --disable-libgme --enable-libgsm --disable-mmal --disable-libmodplug --enabl e-libopus --disable-libilbc --disable-librtmp --disable-libssh --enable-libspeex --enable-librsvg --enable-libvorbis --enable-libvpx --disable-libzvbi --disable-appkit --disable-libbs2b --disable-chromaprint --di sable-libflite --disable-frei0r --disable-libfribidi --enable-fontconfig --disable-ladspa --disable-libass --enable-libfreetype --disable-librubberband --disable-libzmq --disable-libzimg --disable-libsoxr --enabl e-pthreads --enable-libvo-amrwbenc --enable-libmp3lame --disable-libkvazaar --enable-nvenc --disable-libopenh264 --disable-libsnappy --enable-libtheora --disable-libtwolame --enable-libwavpack --enable-libwebp -- enable-libx264 --enable-libx265 --disable-libxvid --disable-armv5te --disable-armv6 --disable-armv6t2 --disable-neon --disable-vfp --disable-vfpv3 --disable-armv8 --disable-mipsdsp --disable-mipsdspr2 --disable-m ipsfpu --disable-altivec --disable-amd3dnow --disable-amd3dnowext --disable-fma4 --disable-xop --enable-pic --cpu=znver1 --enable-lto --disable-doc --disable-htmlpages --enable-manpages Thanks! Manuel