Bug 85132 - ffmpeg runtime segfault with lto
Summary: ffmpeg runtime segfault with lto
Status: RESOLVED INVALID
Alias: None
Product: gcc
Classification: Unclassified
Component: lto (show other bugs)
Version: 8.0.1
: P3 normal
Target Milestone: ---
Assignee: Martin Liška
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-03-30 18:31 UTC by Manuel Lauss
Modified: 2024-10-21 10:12 UTC (History)
1 user (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed: 2018-03-31 00:00:00


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Manuel Lauss 2018-03-30 18:31:41 UTC
ffmpeg-3.4.2 built with gcc-8 as of 20180330 and LTO enabled causes a runtime segfault:

Thread 1 "mpv" received signal SIGSEGV, Segmentation fault.
0x00007ffff686b305 in ff_sine_window_init (window=0x7ffff739a2c0 <ff_sine_960>, n=960) at src/libavcodec/sinewin_tablegen.h:73
73              window[i] = SIN_FIX(sinf((i + 0.5) * (M_PI / (2.0 * n))));
(gdb) bt
#0  0x00007ffff686b305 in ff_sine_window_init (window=0x7ffff739a2c0 <ff_sine_960>, n=960) at src/libavcodec/sinewin_tablegen.h:73
#1  0x00007ffff6861d54 in aac_static_table_init () at src/libavcodec/aacdec_template.c:1142
#2  0x00007ffff511d187 in __pthread_once_slow (once_control=0x7ffff7b4cf20 <aac_table_init.lto_priv>, init_routine=0x7ffff6861256 <aac_static_table_init>) at pthread_once.c:116
#3  0x00007ffff6862aae in aac_decode_init (avctx=0xa5ef00) at src/libavcodec/aacdec_template.c:1159
#4  0x00007ffff6ecbef6 in avcodec_open2 (avctx=0xa5ef00, codec=<optimized out>, options=<optimized out>) at src/libavcodec/utils.c:1020
#5  0x0000000000502b3a in ?? ()
#6  0x00000000004fb7be in ?? ()
#7  0x000000000045afb7 in ?? ()
#8  0x000000000045b8af in ?? ()
#9  0x00000000004d3246 in ?? ()
#10 0x000000000040fa21 in ?? ()
#11 0x00007ffff4c8854b in __libc_start_main (main=0x40f9d0, argc=2, argv=0x7fffffffd1c8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffd1b8)
    at ../csu/libc-start.c:308
#12 0x000000000040fb9a in ?? ()
(gdb) disass
Dump of assembler code for function ff_sine_window_init:
   0x00007ffff686b2c9 <+0>:     push   %r12
   0x00007ffff686b2cb <+2>:     mov    %rdi,%r12
   0x00007ffff686b2ce <+5>:     push   %rbp
   0x00007ffff686b2cf <+6>:     mov    %esi,%ebp
   0x00007ffff686b2d1 <+8>:     push   %rbx
   0x00007ffff686b2d2 <+9>:     xor    %ebx,%ebx
   0x00007ffff686b2d4 <+11>:    cmp    %ebx,%ebp
   0x00007ffff686b2d6 <+13>:    jle    0x7ffff686b310 <ff_sine_window_init+71>
   0x00007ffff686b2d8 <+15>:    vmovsd 0x878108(%rip),%xmm2        # 0x7ffff70e33e8
   0x00007ffff686b2e0 <+23>:    vcvtsi2sd %ebp,%xmm0,%xmm0
   0x00007ffff686b2e4 <+27>:    vaddsd %xmm0,%xmm0,%xmm0
   0x00007ffff686b2e8 <+31>:    vcvtsi2sd %ebx,%xmm1,%xmm1
   0x00007ffff686b2ec <+35>:    vaddsd 0x87811c(%rip),%xmm1,%xmm1        # 0x7ffff70e3410
   0x00007ffff686b2f4 <+43>:    vdivsd %xmm0,%xmm2,%xmm0
   0x00007ffff686b2f8 <+47>:    vmulsd %xmm1,%xmm0,%xmm0
   0x00007ffff686b2fc <+51>:    vcvtsd2ss %xmm0,%xmm0,%xmm0
   0x00007ffff686b300 <+55>:    callq  0x7ffff6801db0 <sinf@plt>
=> 0x00007ffff686b305 <+60>:    vmovss %xmm0,(%r12,%rbx,4)
   0x00007ffff686b30b <+66>:    inc    %rbx
   0x00007ffff686b30e <+69>:    jmp    0x7ffff686b2d4 <ff_sine_window_init+11>
   0x00007ffff686b310 <+71>:    pop    %rbx
   0x00007ffff686b311 <+72>:    pop    %rbp
   0x00007ffff686b312 <+73>:    pop    %r12
   0x00007ffff686b314 <+75>:    retq   
End of assembler dump.
(gdb) info registers
rax            0x0                 0
rbx            0x0                 0
rcx            0x1                 1
rdx            0x78                120
rsi            0x3c0               960
rdi            0x7ffff739a2c0      140737341137600
rbp            0x3c0               0x3c0
rsp            0x7fffffffc840      0x7fffffffc840
r8             0x7fffffffa850      140737488332880
r9             0x7                 7
r10            0xfffffffffffffd07  -761
r11            0x7ffff5682c90      140737310633104
r12            0x7ffff739a2c0      140737341137600
r13            0x7fffffffc9f8      140737488341496
r14            0x0                 0
r15            0x0                 0
rip            0x7ffff686b305      0x7ffff686b305 <ff_sine_window_init+60>
eflags         0x10202             [ IF RF ]
cs             0x33                51
ss             0x2b                43
ds             0x0                 0
es             0x0                 0
fs             0x0                 0
gs             0x0                 0


Without LTO it's fine, the assembly of the faulting function is the same.
I tried to isolate a testcase but failed.

ffmpeg built with:
configure --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --docdir=/usr/share/doc/ffmpeg-3.4.2-r1/html --mandir=/usr/share/man --enable-share
d --cc=x86_64-pc-linux-gnu-gcc --cxx=x86_64-pc-linux-gnu-g++ --ar=x86_64-pc-linux-gnu-ar --optflags=-O3 -ggdb -march=znver1 -mtune=znver1 -flto=16 -fno-fat-lto-objects -fno-strict-aliasing -fexpensive-optimizatio
ns -Wno-deprecated -pipe --disable-static --enable-avfilter --enable-avresample --disable-stripping --enable-nonfree --enable-version3 --disable-indev=alsa --disable-indev=oss --disable-indev=jack --disable-outde
v=alsa --disable-outdev=oss --enable-version3 --enable-version3 --enable-bzlib --disable-runtime-cpudetect --disable-debug --disable-gcrypt --enable-gnutls --enable-gmp --enable-gpl --enable-hardcoded-tables --en
able-iconv --enable-lzma --enable-network --disable-opencl --enable-openssl --enable-postproc --disable-libsmbclient --enable-ffplay --enable-sdl2 --disable-vaapi --enable-vdpau --enable-xlib --enable-libxcb --en
able-libxcb-shm --enable-libxcb-xfixes --enable-zlib --disable-libcdio --disable-libiec61883 --disable-libdc1394 --disable-libcaca --enable-openal --enable-opengl --enable-libv4l2 --enable-libpulse --disable-libd
rm --enable-libopencore-amrwb --enable-libopencore-amrnb --disable-libfdk-aac --enable-libopenjpeg --enable-libbluray --disable-libcelt --disable-libgme --enable-libgsm --disable-mmal --disable-libmodplug --enabl
e-libopus --disable-libilbc --disable-librtmp --disable-libssh --enable-libspeex --enable-librsvg --enable-libvorbis --enable-libvpx --disable-libzvbi --disable-appkit --disable-libbs2b --disable-chromaprint --di
sable-libflite --disable-frei0r --disable-libfribidi --enable-fontconfig --disable-ladspa --disable-libass --enable-libfreetype --disable-librubberband --disable-libzmq --disable-libzimg --disable-libsoxr --enabl
e-pthreads --enable-libvo-amrwbenc --enable-libmp3lame --disable-libkvazaar --enable-nvenc --disable-libopenh264 --disable-libsnappy --enable-libtheora --disable-libtwolame --enable-libwavpack --enable-libwebp --
enable-libx264 --enable-libx265 --disable-libxvid --disable-armv5te --disable-armv6 --disable-armv6t2 --disable-neon --disable-vfp --disable-vfpv3 --disable-armv8 --disable-mipsdsp --disable-mipsdspr2 --disable-m
ipsfpu --disable-altivec --disable-amd3dnow --disable-amd3dnowext --disable-fma4 --disable-xop --enable-pic --cpu=znver1 --enable-lto --disable-doc --disable-htmlpages --enable-manpages

Thanks!
      Manuel
Comment 1 Martin Liška 2018-03-30 19:36:09 UTC
Hi Manual.

Can you please reduce a bit the configure options. The package requires many libraries as dependency. How do you invoke ffmpeg to expose the crash?
Comment 2 Manuel Lauss 2018-03-30 20:33:28 UTC
This is the most minimal ffmpeg config that can trigger the segfault. The interesting switch is "--enabled-hardcoded-tables".  I use the "mpv" Player with a standard mp4 video file to trigger it (https://github.com/mpv-player/mpv/archive/v0.27.2.zip)

configure --prefix=/usr --libdir=/usr/lib64 --shlibdir=/usr/lib64 --docdir=/usr/share/doc/ffmpeg-3.4.2-r1/html --mandir=/usr/share/man --enable-share
d --cc=x86_64-pc-linux-gnu-gcc --cxx=x86_64-pc-linux-gnu-g++ --ar=x86_64-pc-linux-gnu-ar --optflags=-O3 -ggdb -march=znver1 -mtune=znver1 -flto=16 -fno-fat-lto-objects -pipe --disable-static --enable-avfilter --enable-avresample --disable-stripping --disable-encoders --disable-indev=v4l2 --disable-outdev=v4l2 --disable-indev=alsa --disable-indev=oss --disable
-indev=jack --disable-outdev=alsa --disable-outdev=oss --disable-outdev=sdl --disable-bzlib --disable-runtime-cpudetect --disable-debug --disable-gcrypt --disable-gnutls --disable-gmp --enable-gpl --enable-hardco
ded-tables --disable-iconv --disable-lzma --disable-network --disable-opencl --disable-openssl --disable-postproc --disable-libsmbclient --disable-ffplay --disable-sdl2 --disable-vaapi --disable-vdpau --disable-x
lib --disable-libxcb --disable-libxcb-shm --disable-libxcb-xfixes --disable-zlib --disable-libcdio --disable-libiec61883 --disable-libdc1394 --disable-libcaca --disable-openal --disable-opengl --disable-libv4l2 -
-disable-libpulse --disable-libdrm --disable-libopencore-amrwb --disable-libopencore-amrnb --disable-libfdk-aac --disable-libopenjpeg --disable-libbluray --disable-libcelt --disable-libgme --disable-libgsm --disa
ble-mmal --disable-libmodplug --disable-libopus --disable-libilbc --disable-librtmp --disable-libssh --disable-libspeex --disable-librsvg --disable-libvorbis --disable-libvpx --disable-libzvbi --disable-appkit --
disable-libbs2b --disable-chromaprint --disable-libflite --disable-frei0r --disable-libfribidi --disable-fontconfig --disable-ladspa --disable-libass --disable-libfreetype --disable-librubberband --disable-libzmq
 --disable-libzimg --disable-libsoxr --enable-pthreads --disable-armv5te --disable-armv6 --disable-armv6t2 --disable-neon --disable-vfp --disable-vfpv3 --disable-armv8 --disable-mipsdsp --disable-mipsdspr2 --disa
ble-mipsfpu --disable-altivec --disable-amd3dnow --disable-amd3dnowext --disable-fma4 --disable-xop --cpu=znver1 --enable-lto --disable-doc --disable-htmlpages --enable-manpages
Comment 3 Martin Liška 2018-03-31 08:00:56 UTC
(In reply to Manuel Lauss from comment #2)
> This is the most minimal ffmpeg config that can trigger the segfault. The
> interesting switch is "--enabled-hardcoded-tables".  I use the "mpv" Player
> with a standard mp4 video file to trigger it
> (https://github.com/mpv-player/mpv/archive/v0.27.2.zip)

Good. so we have minimal set of options for ffmpeg. Now please explain me how to build the MPV tool? Is it really needed? Can't one trigger that directly with ffmepg? Please upload somewhere the mp4 file so that I can reproduce it.
Comment 4 Manuel Lauss 2018-03-31 08:19:14 UTC
(In reply to Martin Liška from comment #3)
> (In reply to Manuel Lauss from comment #2)
> > This is the most minimal ffmpeg config that can trigger the segfault. The
> > interesting switch is "--enabled-hardcoded-tables".  I use the "mpv" Player
> > with a standard mp4 video file to trigger it
> > (https://github.com/mpv-player/mpv/archive/v0.27.2.zip)
> 
> Good. so we have minimal set of options for ffmpeg. Now please explain me
> how to build the MPV tool? Is it really needed? Can't one trigger that
> directly with ffmepg? Please upload somewhere the mp4 file so that I can
> reproduce it.

The segfault happens during initialization of ffmpeg when mpv player starts.  I can also trigger it with firefox on e.g. youtube: it crashes the tab.


ad mpv: unpack, configure with the following line (you need python3):
CCFLAGS="-O3 -march=znver1 -mtune=znver1 -flto=16 -fno-fat-lto-objects -pipe" LINKFLAGS="-O3 -march=znver1 -mtune=znver1 -flto=16 -fno-fat-lto-objects -pipe -Wl,--no-as-needed" ./waf --prefix=/usr --libdir=/usr/lib64 --confdir=/etc/mpv --docdir=/usr/share/doc/mpv-0.27.2 --htmldir=/usr/share/doc/mpv-0.27.2/html --enable-libmpv-shared --disable-libmpv-static --disable-static-build --disable-optimize --disable-debug-build --enable-html-build --disable-pdf-build --enable-cplugins --disable-zsh-comp --disable-test --disable-android --enable-iconv --disable-libsmbclient --enable-lua --lua=luajit --disable-javascript --enable-x11  --disable-build-date configure

./waf build
Comment 5 Martin Liška 2018-03-31 08:41:18 UTC
(In reply to Manuel Lauss from comment #4)
> (In reply to Martin Liška from comment #3)
> > (In reply to Manuel Lauss from comment #2)
> > > This is the most minimal ffmpeg config that can trigger the segfault. The
> > > interesting switch is "--enabled-hardcoded-tables".  I use the "mpv" Player
> > > with a standard mp4 video file to trigger it
> > > (https://github.com/mpv-player/mpv/archive/v0.27.2.zip)
> > 
> > Good. so we have minimal set of options for ffmpeg. Now please explain me
> > how to build the MPV tool? Is it really needed? Can't one trigger that
> > directly with ffmepg? Please upload somewhere the mp4 file so that I can
> > reproduce it.
> 
> The segfault happens during initialization of ffmpeg when mpv player starts.
> I can also trigger it with firefox on e.g. youtube: it crashes the tab.

Ok, so please download following sample video:
http://techslides.com/demos/sample-videos/small.mp4

And try to run ffmpeg to print info:
./ffmpeg -i small.mp4

Does it also segfault?

> 
> 
> ad mpv: unpack, configure with the following line (you need python3):
> CCFLAGS="-O3 -march=znver1 -mtune=znver1 -flto=16 -fno-fat-lto-objects
> -pipe" LINKFLAGS="-O3 -march=znver1 -mtune=znver1 -flto=16
> -fno-fat-lto-objects -pipe -Wl,--no-as-needed" ./waf --prefix=/usr
> --libdir=/usr/lib64 --confdir=/etc/mpv --docdir=/usr/share/doc/mpv-0.27.2
> --htmldir=/usr/share/doc/mpv-0.27.2/html --enable-libmpv-shared
> --disable-libmpv-static --disable-static-build --disable-optimize
> --disable-debug-build --enable-html-build --disable-pdf-build
> --enable-cplugins --disable-zsh-comp --disable-test --disable-android
> --enable-iconv --disable-libsmbclient --enable-lua --lua=luajit
> --disable-javascript --enable-x11  --disable-build-date configure
> 
> ./waf build

How the player knows where to find the built ffmpeg libraries?
Comment 6 Manuel Lauss 2018-03-31 08:53:22 UTC
(In reply to Martin Liška from comment #5)
> (In reply to Manuel Lauss from comment #4)
> > (In reply to Martin Liška from comment #3)
> > > (In reply to Manuel Lauss from comment #2)
> > > > This is the most minimal ffmpeg config that can trigger the segfault. The
> > > > interesting switch is "--enabled-hardcoded-tables".  I use the "mpv" Player
> > > > with a standard mp4 video file to trigger it
> > > > (https://github.com/mpv-player/mpv/archive/v0.27.2.zip)
> > > 
> > > Good. so we have minimal set of options for ffmpeg. Now please explain me
> > > how to build the MPV tool? Is it really needed? Can't one trigger that
> > > directly with ffmepg? Please upload somewhere the mp4 file so that I can
> > > reproduce it.
> > 
> > The segfault happens during initialization of ffmpeg when mpv player starts.
> > I can also trigger it with firefox on e.g. youtube: it crashes the tab.
> 
> Ok, so please download following sample video:
> http://techslides.com/demos/sample-videos/small.mp4
> 
> And try to run ffmpeg to print info:
> ./ffmpeg -i small.mp4
> 
> Does it also segfault?

Yes, same backtrace, same fault location.
Comment 7 Martin Liška 2018-03-31 09:06:18 UTC
(In reply to Manuel Lauss from comment #6)
> (In reply to Martin Liška from comment #5)
> > (In reply to Manuel Lauss from comment #4)
> > > (In reply to Martin Liška from comment #3)
> > > > (In reply to Manuel Lauss from comment #2)
> > > > > This is the most minimal ffmpeg config that can trigger the segfault. The
> > > > > interesting switch is "--enabled-hardcoded-tables".  I use the "mpv" Player
> > > > > with a standard mp4 video file to trigger it
> > > > > (https://github.com/mpv-player/mpv/archive/v0.27.2.zip)
> > > > 
> > > > Good. so we have minimal set of options for ffmpeg. Now please explain me
> > > > how to build the MPV tool? Is it really needed? Can't one trigger that
> > > > directly with ffmepg? Please upload somewhere the mp4 file so that I can
> > > > reproduce it.
> > > 
> > > The segfault happens during initialization of ffmpeg when mpv player starts.
> > > I can also trigger it with firefox on e.g. youtube: it crashes the tab.
> > 
> > Ok, so please download following sample video:
> > http://techslides.com/demos/sample-videos/small.mp4
> > 
> > And try to run ffmpeg to print info:
> > ./ffmpeg -i small.mp4
> > 
> > Does it also segfault?
> 
> Yes, same backtrace, same fault location.

Hm, I put __builtin_abort to:
static av_cold void aac_static_table_init(void)

function. And it's unfortunately called. Probably because:
$ ./ffmpeg -i small.mp4 test.mp3
...
Decoder (codec aac) not found for input stream #0:1

Are you sure the configure options enable the codec (AAC)?
Comment 8 Martin Liška 2018-03-31 09:47:38 UTC
Stupid mistake from me. Now I can reproduce that:

LD_LIBRARY_PATH=./libavcodec/ gdb --args ./ffmpeg -i small.mp4
Comment 9 Martin Liška 2018-03-31 10:15:06 UTC
So it's really caused by --enable-hardcoded-tables. With that you have in sinewin.c pre-processed source file:

extern const INTFLOAT __attribute__ ((aligned (32))) ff_sine_960[960];

As you can se, it's const (a.k.a. READONLY). Thus doing a write to it in ff_sine_window_init causes a segfault.

Thus closing as invalid.
Comment 10 Manuel Lauss 2018-03-31 13:35:26 UTC
Built with gcc-7.3.0 and LTO it works fine; is this a bug in 7.3.0 instead then?
Comment 11 Martin Liška 2018-03-31 17:27:02 UTC
It was changed on trunk in:

commit 371072bf395be11f36ef31bb3cfec06bbfc58597
Author: law <law@138bc75d-0d04-0410-961f-82ee72b054a4>
Date:   Fri Sep 1 16:26:00 2017 +0000

            * varasm.c (bss_initializer_p): Do not put constants into .bss
            (categorize_decl_for_section): Handle bss_initializer_p returning
            false when DECL_INITIAL is NULL.
    
            * gcc.target/i386/const-in-bss.c: New test.
    
    git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@251602 138bc75d-0d04-0410-961f-82ee72b054a4

Note the using an undefined behavior gives optimizer opportunity to optimize to any way. In this case GCC 7 generated working and GCC 8 not working code in case of ffmpeg.
Comment 12 Carl Eugen Hoyos 2018-10-19 18:42:14 UTC
Should be fixed:
http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=10f46815
https://trac.ffmpeg.org/ticket/7491

I wonder a little how the FFmpeg developers were supposed to find out about this bug...
Comment 13 Martin Liška 2018-10-22 09:23:42 UTC
(In reply to Carl Eugen Hoyos from comment #12)
> Should be fixed:
> http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=10f46815
> https://trac.ffmpeg.org/ticket/7491
> 
> I wonder a little how the FFmpeg developers were supposed to find out about
> this bug...

Good, thanks for mainline fix!