[Bug c/109137] New: [12/13 regression] Compiling ffmpeg with -m32 on x86_64-pc-linux-gnu hangs on libavcodec/h264_cabac.c since r12-9086-g489c81db7d4f75

sam at gentoo dot org gcc-bugzilla@gcc.gnu.org
Wed Mar 15 00:30:46 GMT 2023 

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109137

            Bug ID: 109137
           Summary: [12/13 regression] Compiling ffmpeg with -m32 on
                    x86_64-pc-linux-gnu hangs on libavcodec/h264_cabac.c
                    since r12-9086-g489c81db7d4f75
           Product: gcc
           Version: 13.0
               URL: https://bugs.gentoo.org/900937
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c
          Assignee: unassigned at gcc dot gnu.org
          Reporter: sam at gentoo dot org
                CC: hubicka at gcc dot gnu.org
  Target Milestone: ---

Building ffmpeg for multilib (32-bit x86 on amd64, i.e. x86_64-pc-linux-gnu
-m32) hangs when using -O3 -march=znver1 on libavcodec/h264_cabac.c.

Notes:
- Originally reported downstream in Gentoo at https://bugs.gentoo.org/900937.
- 12.2.1 20230121 is good
- 12.2.1 20230304 is bad
- 13.0.1 20230312 is bad
- Seems to bisect to r12-9086-g489c81db7d4f75
- Needs CFLAGS="-O3 -march=znver1" to trigger

The bisect result 489c81db7d4f75894e9d34aa90fe7224cfafb53a
(r12-9086-g489c81db7d4f75) doesn't revert cleanly on tip of releases/gcc-12 so
I haven't been able to confirm that bit yet.

>From `ps faux`:
```
sam      3394612  5.1  0.0  23596 19736 pts/11   SN+  00:19   0:02  |       |  
            \_ make -j32 -l32 V=1
sam      3399918  0.0  0.0   8180  3612 pts/11   SN+  00:20   0:00  |       |  
                \_ //usr/bin/x86_64-pc-linux-gnu-gcc -m32 -mfpmath=sse -I.
-Isrc/ -D_ISOC99_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
-D_POSIX_C_SOURCE=200112 -D_XOPEN_SOURCE=600 -DPIC -DHAVE_AV_CONFIG_H
-DBUILDING_avcodec -march=znver1 -O3 -Werror=implicit-function-declaration
-Werror=implicit-int -Wreturn-type -Wformat -Wint-conversion -Waddress
-Warray-bounds -Wfree-nonheap-object -Wimplicit-function-declaration
-Wimplicit-int -Wincompatible-pointer-types -Wint-conversion
-Wint-to-pointer-cast -Wmain -Wnonnull -Wodr -Wparentheses -Wreturn-type
-Wsizeof-pointer-memaccess -Wstrict-aliasing -Wstring-compare -Wuninitialized
-Wunused-value -Wvarargs -march=znver1 -std=c11 -fPIC
-Wdeclaration-after-statement -Wall -Wdisabled-optimization -Wpointer-arith
-Wredundant-decls -Wwrite-strings -Wtype-limits -Wundef -Wmissing-prototypes
-Wno-pointer-to-int-cast -Wstrict-prototypes -Wempty-body -Wno-parentheses
-Wno-switch -Wno-format-zero-length -Wno-pointer-sign
-Wno-unused-const-variable -Wno-bool-operation -Wno-char-subscripts
-march=znver1 -O3 -Werror=implicit-function-declaration -Werror=implicit-int
-Wreturn-type -Wformat -Wint-conversion -Waddress -Warray-bounds
-Wfree-nonheap-object -Wimplicit-function-declaration -Wimplicit-int
-Wincompatible-pointer-types -Wint-conversion -Wint-to-pointer-cast -Wmain
-Wnonnull -Wodr -Wparentheses -Wreturn-type -Wsizeof-pointer-memaccess
-Wstrict-aliasing -Wstring-compare -Wuninitialized -Wunused-value -Wvarargs
-fno-math-errno -fno-signed-zeros -fno-tree-vectorize -Werror=format-security
-Werror=implicit-function-declaration -Werror=missing-prototypes
-Werror=return-type -Werror=vla -Wformat -Wno-maybe-uninitialized -MMD -MF
libavcodec/h264_cabac.d -MT libavcodec/h264_cabac.o -c -o
libavcodec/h264_cabac.o src/libavcodec/h264_cabac.c
sam      3399921  100  0.1 109632 88364 pts/11   RN+  00:20   0:43  |       |  
                    \_ /usr/libexec/gcc/x86_64-pc-linux-gnu/13/cc1 -quiet -I .
-I src/ -imultilib 32 -MMD libavcodec/h264_cabac.d -MF libavcodec/h264_cabac.d
-MT libavcodec/h264_cabac.o -D _ISOC99_SOURCE -D _FILE_OFFSET_BITS=64 -D
_LARGEFILE_SOURCE -D _POSIX_C_SOURCE=200112 -D _XOPEN_SOURCE=600 -D PIC -D
HAVE_AV_CONFIG_H -D BUILDING_avcodec src/libavcodec/h264_cabac.c -quiet
-dumpdir libavcodec/ -dumpbase h264_cabac.c -dumpbase-ext .c -m32 -mfpmath=sse
-march=znver1 -O3 -O3 -Werror=implicit-function-declaration
-Werror=implicit-int -Wformat=1 -Warray-bounds=1 -Wdeclaration-after-statement
-Wall -Wdisabled-optimization -Wpointer-arith -Wredundant-decls -Wwrite-strings
-Wtype-limits -Wundef -Wmissing-prototypes -Wno-pointer-to-int-cast
-Wstrict-prototypes -Wempty-body -Wno-switch -Wno-format-zero-length
-Wno-pointer-sign -Wunused-const-variable=0 -Wno-bool-operation
-Wno-char-subscripts -Werror=implicit-function-declaration -Werror=implicit-int
-Wformat=1 -Waddress -Warray-bounds=1 -Wfree-nonheap-object
-Wimplicit-function-declaration -Wimplicit-int -Wincompatible-pointer-types
-Wint-conversion -Wint-to-pointer-cast -Wmain -Wnonnull -Wodr -Wparentheses
-Wreturn-type -Wsizeof-pointer-memaccess -Wstrict-aliasing -Wstring-compare
-Wuninitialized -Wunused-value -Wvarargs -Werror=format-security
-Werror=implicit-function-declaration -Werror=missing-prototypes
-Werror=return-type -Werror=vla -Wformat=1 -Wno-maybe-uninitialized -std=c11
-fPIC -fno-math-errno -fno-signed-zeros -fno-tree-vectorize -o
/var/tmp/portage/media-video/ffmpeg-4.4.3/temp/cc7EadEm.s
```

---

Now, if I try to run it by itself, I get:
```
# /usr/bin/x86_64-pc-linux-gnu-gcc-12 -m32 -mfpmath=sse -I. -Isrc/
-D_ISOC99_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
-D_POSIX_C_SOURCE=200112 -D_XOPEN_SOURCE=600 -DPIC -DHAVE_AV_CONFIG_H
-DBUILDING_avcodec -march=znver1 -O3 -Werror=implicit-function-declaration
-Werror=implicit-int -Wreturn-type -Wformat -Wint-conversion -Waddress
-Warray-bounds -Wfree-nonheap-object -Wimplicit-function-declaration
-Wimplicit-int -Wincompatible-pointer-types -Wint-conversion
-Wint-to-pointer-cast -Wmain -Wnonnull -Wodr -Wparentheses -Wreturn-type
-Wsizeof-pointer-memaccess -Wstrict-aliasing -Wstring-compare -Wuninitialized
-Wunused-value -Wvarargs -march=znver1 -std=c11 -fPIC
-Wdeclaration-after-statement -Wall -Wdisabled-optimization -Wpointer-arith
-Wredundant-decls -Wwrite-strings -Wtype-limits -Wundef -Wmissing-prototypes
-Wno-pointer-to-int-cast -Wstrict-prototypes -Wempty-body -Wno-parentheses
-Wno-switch -Wno-format-zero-length -Wno-pointer-sign
-Wno-unused-const-variable -Wno-bool-operation -Wno-char-subscripts
-march=znver1 -O3 -Werror=implicit-function-declaration -Werror=implicit-int
-Wreturn-type -Wformat -Wint-conversion -Waddress -Warray-bounds
-Wfree-nonheap-object -Wimplicit-function-declaration -Wimplicit-int
-Wincompatible-pointer-types -Wint-conversion -Wint-to-pointer-cast -Wmain
-Wnonnull -Wodr -Wparentheses -Wreturn-type -Wsizeof-pointer-memaccess
-Wstrict-aliasing -Wstring-compare -Wuninitialized -Wunused-value -Wvarargs
-fno-math-errno -fno-signed-zeros -fno-tree-vectorize -Werror=format-security
-Werror=implicit-function-declaration -Werror=missing-prototypes
-Werror=return-type -Werror=vla -Wformat -Wno-maybe-uninitialized -MMD -MF
libavcodec/h264_cabac.d -MT libavcodec/h264_cabac.o -c -o
libavcodec/h264_cabac.o src/libavcodec/h264_cabac.c
In file included from src/libavcodec/cabac_functions.h:47,
                 from src/libavcodec/h264_cabac.c:36:
In function 'get_cabac_inline_x86',
    inlined from 'get_cabac' at src/libavcodec/cabac_functions.h:140:12,
    inlined from 'decode_cabac_mb_intra4x4_pred_mode' at
src/libavcodec/h264_cabac.c:1368:9,
    inlined from 'ff_h264_decode_mb_cabac' at
src/libavcodec/h264_cabac.c:2072:32:
src/libavcodec/x86/cabac.h:194:5: error: 'asm' operand has impossible
constraints
  194 |     __asm__ volatile(
      |     ^~~~~~~
src/libavcodec/x86/cabac.h:194:5: error: 'asm' operand has impossible
constraints
src/libavcodec/x86/cabac.h:194:5: error: 'asm' operand has impossible
constraints
src/libavcodec/x86/cabac.h:194:5: error: 'asm' operand has impossible
constraints
[... hangs here ...]
```

But with GCC 11, it completes very quickly (1s or so, or less) with no errors.

More information about the Gcc-bugs mailing list