Hi, We are working on upgrading Python 3.8 to 3.9 in Fedora Rawhide. Two builds ago, the build was fine on the 3 archs (x86-64, aarch64, ppc64), but when GCC has been upgraded from GCC 9.2.1 to GCC 10.0.1, we started the get the following error on ppc64le (and only on this arch, the two others are building successfully): /tmp/cc7LZ1xH.s: Assembler messages: /tmp/cc7LZ1xH.s:91549: Error: redefined symbol cannot be used on reloc /tmp/cc7LZ1xH.s:275099: Error: redefined symbol cannot be used on reloc lto-wrapper: fatal error: gcc returned 1 exit status compilation terminated. /usr/bin/ld: error: lto-wrapper failed collect2: error: ld returned 1 exit status Full build logs: https://kojipkgs.fedoraproject.org//work/tasks/606/40790606/build.log We build Python using LTO and PGO. The first step of PGO (-fprofile-generate) went fine, but the second step (-fprofile-use) is the one which failed. GCC error message is unclear :-( I'm not sure if the error comes from the build of a single .c file, or on the linking command creating libpython3.9.so from all .o files. A Python .c file is built with a command like (that's the second PGO step, using -fprofile-use -fprofile-correction): --- gcc -pthread -c -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mcpu=power8 -mtune=power8 -fasynchronous-unwind-tables -fstack-clash-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mcpu=power8 -mtune=power8 -fasynchronous-unwind-tables -fstack-clash-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mcpu=power8 -mtune=power8 -fasynchronous-unwind-tables -fstack-clash-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mcpu=power8 -mtune=power8 -fasynchronous-unwind-tables -fstack-clash-protection -D_GNU_SOURCE -fPIC -fwrapv -fno-semantic-interposition -flto -fuse-linker-plugin -ffat-lto-objects -flto-partition=none -g -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -fvisibility=hidden -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mcpu=power8 -mtune=power8 -fasynchronous-unwind-tables -fstack-clash-protection -D_GNU_SOURCE -fPIC -fwrapv -fno-semantic-interposition -fprofile-use -fprofile-correction -I/builddir/build/BUILD/Python-3.9.0a2/Include/internal -IObjects -IInclude -IPython -I. -I/builddir/build/BUILD/Python-3.9.0a2/Include -fPIC -DPy_BUILD_CORE -o Python/Python-ast.o /builddir/build/BUILD/Python-3.9.0a2/Python/Python-ast.c --- Link libpython3.9.so: --- gcc -pthread -shared -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -g -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -g -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -fno-semantic-interposition -g -flto -fuse-linker-plugin -ffat-lto-objects -flto-partition=none -g -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -fno-semantic-interposition -g -o libpython3.9.so -lcrypt -lpthread -ldl -lutil -lm -lm Modules/getbuildinfo.o Parser/acceler.o Parser/grammar1.o Parser/listnode.o Parser/node.o Parser/parser.o Parser/token.o Parser/myreadline.o Parser/parsetok.o Parser/tokenizer.o Objects/abstract.o Objects/accu.o Objects/boolobject.o Objects/bytes_methods.o Objects/bytearrayobject.o Objects/bytesobject.o Objects/call.o Objects/capsule.o Objects/cellobject.o Objects/classobject.o Objects/codeobject.o Objects/complexobject.o Objects/descrobject.o Objects/enumobject.o Objects/exceptions.o Objects/genobject.o Objects/fileobject.o Objects/floatobject.o Objects/frameobject.o Objects/funcobject.o Objects/interpreteridobject.o Objects/iterobject.o Objects/listobject.o Objects/longobject.o Objects/dictobject.o Objects/odictobject.o Objects/memoryobject.o Objects/methodobject.o Objects/moduleobject.o Objects/namespaceobject.o Objects/object.o Objects/obmalloc.o Objects/picklebufobject.o Objects/rangeobject.o Objects/setobject.o Objects/sliceobject.o Objects/structseq.o Objects/tupleobject.o Objects/typeobject.o Objects/unicodeobject.o Objects/unicodectype.o Objects/weakrefobject.o Python/_warnings.o Python/Python-ast.o Python/asdl.o Python/ast.o Python/ast_opt.o Python/ast_unparse.o Python/bltinmodule.o Python/ceval.o Python/codecs.o Python/compile.o Python/context.o Python/dynamic_annotations.o Python/errors.o Python/frozenmain.o Python/future.o Python/getargs.o Python/getcompiler.o Python/getcopyright.o Python/getplatform.o Python/getversion.o Python/graminit.o Python/hamt.o Python/import.o Python/importdl.o Python/initconfig.o Python/marshal.o Python/modsupport.o Python/mysnprintf.o Python/mystrtoul.o Python/pathconfig.o Python/peephole.o Python/preconfig.o Python/pyarena.o Python/pyctype.o Python/pyfpe.o Python/pyhash.o Python/pylifecycle.o Python/pymath.o Python/pystate.o Python/pythonrun.o Python/pytime.o Python/bootstrap_hash.o Python/structmember.o Python/symtable.o Python/sysmodule.o Python/thread.o Python/traceback.o Python/getopt.o Python/pystrcmp.o Python/pystrtod.o Python/pystrhex.o Python/dtoa.o Python/formatter_unicode.o Python/fileutils.o Python/dynload_shlib.o Python/pydtrace.o Modules/config.o Modules/getpath.o Modules/main.o Modules/gcmodule.o Modules/posixmodule.o Modules/errnomodule.o Modules/pwdmodule.o Modules/_sre.o Modules/_codecsmodule.o Modules/_weakref.o Modules/_functoolsmodule.o Modules/_operator.o Modules/_collectionsmodule.o Modules/_abc.o Modules/itertoolsmodule.o Modules/atexitmodule.o Modules/signalmodule.o Modules/_stat.o Modules/timemodule.o Modules/_threadmodule.o Modules/_localemodule.o Modules/_iomodule.o Modules/iobase.o Modules/fileio.o Modules/bytesio.o Modules/bufferedio.o Modules/textio.o Modules/stringio.o Modules/faulthandler.o Modules/_tracemalloc.o Modules/hashtable.o Modules/symtablemodule.o Modules/xxsubtype.o Python/frozen.o --- Python configure options: --- ./configure --build=ppc64le-redhat-linux-gnu --host=ppc64le-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --enable-ipv6 --enable-shared --with-computed-gotos=yes --with-dbmliborder=gdbm:ndbm:bdb --with-system-expat --with-system-ffi --enable-loadable-sqlite-extensions --with-dtrace --with-lto --with-ssl-default-suites=openssl --with-valgrind --without-ensurepip --enable-optimizations --- I found the error message in GNU gas assembler: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=blob;f=gas/write.c;h=5d9a2394224d0c36129cc70f0d5755c7014ff9bb;hb=HEAD#l1169 The error message is quite old, it has been added in 2008: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commitdiff;h=76d1293995586b060a2dc642a72c877db7092580;hp=b96e292732664fd8c63970ca11c3a9ac92f60ff9 -- Fedora builds of python39: https://koschei.fedoraproject.org/package/python39 Build 40639489 is green, build 40790556 is red. One interesting difference is GCC 9.2.1 upgraded to GCC 10.0.1. All differences (Dependency changes from previous build): glibc-devel 2.30.9000-29.fc32 2.30.9000-30.fc32 1 redhat-rpm-config 146-1.fc32 148-1.fc32 1 gcc-c++ 9.2.1-1.fc32.3 10.0.1-0.3.fc32 1 glibc-all-langpacks 2.30.9000-29.fc32 2.30.9000-30.fc32 1 sqlite-devel 3.30.1-2.fc32 3.30.1-3.fc32 1 glibc-common 2.30.9000-29.fc32 2.30.9000-30.fc32 2 perl-Exporter 5.73-440.fc31 5.74-1.fc32 2 gcc 9.2.1-1.fc32.3 10.0.1-0.3.fc32 2 annobin 8.92-1.fc32 9.01-1.fc32 2 sqlite 3.30.1-2.fc32 3.30.1-3.fc32 2 libxcrypt-devel 4.4.10-2.fc32 4.4.11-1.fc32 2 libstdc++-devel 9.2.1-1.fc32.3 10.0.1-0.3.fc32 2 libgcc 9.2.1-1.fc32.3 10.0.1-0.3.fc32 2 glibc 2.30.9000-29.fc32 2.30.9000-30.fc32 2 sqlite-libs 3.30.1-2.fc32 3.30.1-3.fc32 2 libxcrypt 4.4.10-2.fc32 4.4.11-1.fc32 2 glibc-headers 2.30.9000-29.fc32 2.30.9000-30.fc32 2 libstdc++ 9.2.1-1.fc32.3 10.0.1-0.3.fc32 2 fontconfig-devel 2.13.92-3.fc32 2.13.92-4.fc32 3 cpp 9.2.1-1.fc32.3 10.0.1-0.3.fc32 3 tzdata 2019c-1.fc32 2019c-2.fc32 3 kernel-headers 5.5.0-0.rc6.git0.1.... 5.5.0-0.rc7.git0.1.... 3 libgomp 9.2.1-1.fc32.3 10.0.1-0.3.fc32 3 fontconfig 2.13.92-3.fc32 2.13.92-4.fc32 3 glibc-minimal-langpack 2.30.9000-29.fc32 2.30.9000-30.fc32 3 libtool-ltdl 2.4.6-31.fc31 2.4.6-32.fc32
>GCC error message is unclear Well not is not a GCC error message but rather the assembler is producing the error message.
Can you use -save-temps and look at the generated assembler file in around those two lines? This could be some inline-asm that causes the error too.
I agree with Andrew. Please attach the requested file so we can have a look.
> Can you use -save-temps and look at the generated assembler file in around those two lines? This could be some inline-asm that causes the error too. The error occurs on linking .o files into libpython3.9.so when at the second step of the profile guided optimization (PGO); Python is rebuilt using the collected profiles. I added -save-temps and gcc created a libpython3.9.so.lto_wrapper_args file: --- -fresolution=-lm.res -flinker-output=dyn Modules/getbuildinfo.o Parser/acceler.o Parser/grammar1.o Parser/listnode.o Parser/node.o Parser/parser.o Parser/token.o Parser/myreadline.o Parser/parsetok.o Parser/tokenizer.o Objects/abstract.o Objects/accu.o Objects/boolobject.o Objects/bytes_methods.o Objects/bytearrayobject.o Objects/bytesobject.o Objects/call.o Objects/capsule.o Objects/cellobject.o Objects/classobject.o Objects/codeobject.o Objects/complexobject.o Objects/descrobject.o Objects/enumobject.o Objects/exceptions.o Objects/genobject.o Objects/fileobject.o Objects/floatobject.o Objects/frameobject.o Objects/funcobject.o Objects/interpreteridobject.o Objects/iterobject.o Objects/listobject.o Objects/longobject.o Objects/dictobject.o Objects/odictobject.o Objects/memoryobject.o Objects/methodobject.o Objects/moduleobject.o Objects/namespaceobject.o Objects/object.o Objects/obmalloc.o Objects/picklebufobject.o Objects/rangeobject.o Objects/setobject.o Objects/sliceobject.o Objects/structseq.o Objects/tupleobject.o Objects/typeobject.o Objects/unicodeobject.o Objects/unicodectype.o Objects/weakrefobject.o Python/_warnings.o Python/Python-ast.o Python/asdl.o Python/ast.o Python/ast_opt.o Python/ast_unparse.o Python/bltinmodule.o Python/ceval.o Python/codecs.o Python/compile.o Python/context.o Python/dynamic_annotations.o Python/errors.o Python/frozenmain.o Python/future.o Python/getargs.o Python/getcompiler.o Python/getcopyright.o Python/getplatform.o Python/getversion.o Python/graminit.o Python/hamt.o Python/import.o Python/importdl.o Python/initconfig.o Python/marshal.o Python/modsupport.o Python/mysnprintf.o Python/mystrtoul.o Python/pathconfig.o Python/peephole.o Python/preconfig.o Python/pyarena.o Python/pyctype.o Python/pyfpe.o Python/pyhash.o Python/pylifecycle.o Python/pymath.o Python/pystate.o Python/pythonrun.o Python/pytime.o Python/bootstrap_hash.o Python/structmember.o Python/symtable.o Python/sysmodule.o Python/thread.o Python/traceback.o Python/getopt.o Python/pystrcmp.o Python/pystrtod.o Python/pystrhex.o Python/dtoa.o Python/formatter_unicode.o Python/fileutils.o Python/dynload_shlib.o Modules/config.o Modules/getpath.o Modules/main.o Modules/gcmodule.o Modules/posixmodule.o Modules/errnomodule.o Modules/pwdmodule.o Modules/_sre.o Modules/_codecsmodule.o Modules/_weakref.o Modules/_functoolsmodule.o Modules/_operator.o Modules/_collectionsmodule.o Modules/_abc.o Modules/itertoolsmodule.o Modules/atexitmodule.o Modules/signalmodule.o Modules/_stat.o Modules/timemodule.o Modules/_threadmodule.o Modules/_localemodule.o Modules/_iomodule.o Modules/iobase.o Modules/fileio.o Modules/bytesio.o Modules/bufferedio.o Modules/textio.o Modules/stringio.o Modules/faulthandler.o Modules/_tracemalloc.o Modules/hashtable.o Modules/symtablemodule.o Modules/xxsubtype.o Python/frozen.o --- Processes running when the bug occurs: PID TTY STAT TIME COMMAND 19877 pts/0 T 0:00 gcc -save-temps -pthread -shared -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -g -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -g -Wl,-z,relro -Wl,--a 19878 pts/0 T 0:00 /usr/libexec/gcc/ppc64le-redhat-linux/10/collect2 -plugin /usr/libexec/gcc/ppc64le-redhat-linux/10/liblto_plugin.so -plugin-opt=/usr/libexe 19879 pts/0 T 0:00 /usr/bin/ld -plugin /usr/libexec/gcc/ppc64le-redhat-linux/10/liblto_plugin.so -plugin-opt=/usr/libexec/gcc/ppc64le-redhat-linux/10/lto-wrap 19880 pts/0 T 0:00 /usr/libexec/gcc/ppc64le-redhat-linux/10/lto-wrapper @libpython3.9.so.lto_wrapper_args 19881 pts/0 T 0:00 gcc @/tmp/ccRJQGYo 19882 pts/0 T 0:55 /usr/libexec/gcc/ppc64le-redhat-linux/10/lto1 -quiet -dumpdir ./ -dumpbase libpython3.9.so -msecure-plt -mcpu=power8 -mtune=power8 -mcpu=po
$ gcc -save-temps -pthread -shared -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -g -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -g -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -fno-semantic-interposition -g -flto -fuse-linker-plugin -ffat-lto-objects -flto-partition=none -g -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -fno-semantic-interposition -g -o libpython3.9.so Modules/getbuildinfo.o Parser/acceler.o Parser/grammar1.o Parser/listnode.o Parser/node.o Parser/parser.o Parser/token.o Parser/myreadline.o Parser/parsetok.o Parser/tokenizer.o Objects/abstract.o Objects/accu.o Objects/boolobject.o Objects/bytes_methods.o Objects/bytearrayobject.o Objects/bytesobject.o Objects/call.o Objects/capsule.o Objects/cellobject.o Objects/classobject.o Objects/codeobject.o Objects/complexobject.o Objects/descrobject.o Objects/enumobject.o Objects/exceptions.o Objects/genobject.o Objects/fileobject.o Objects/floatobject.o Objects/frameobject.o Objects/funcobject.o Objects/interpreteridobject.o Objects/iterobject.o Objects/listobject.o Objects/longobject.o Objects/dictobject.o Objects/odictobject.o Objects/memoryobject.o Objects/methodobject.o Objects/moduleobject.o Objects/namespaceobject.o Objects/object.o Objects/obmalloc.o Objects/picklebufobject.o Objects/rangeobject.o Objects/setobject.o Objects/sliceobject.o Objects/structseq.o Objects/tupleobject.o Objects/typeobject.o Objects/unicodeobject.o Objects/unicodectype.o Objects/weakrefobject.o Python/_warnings.o Python/Python-ast.o Python/asdl.o Python/ast.o Python/ast_opt.o Python/ast_unparse.o Python/bltinmodule.o Python/ceval.o Python/codecs.o Python/compile.o Python/context.o Python/dynamic_annotations.o Python/errors.o Python/frozenmain.o Python/future.o Python/getargs.o Python/getcompiler.o Python/getcopyright.o Python/getplatform.o Python/getversion.o Python/graminit.o Python/hamt.o Python/import.o Python/importdl.o Python/initconfig.o Python/marshal.o Python/modsupport.o Python/mysnprintf.o Python/mystrtoul.o Python/pathconfig.o Python/peephole.o Python/preconfig.o Python/pyarena.o Python/pyctype.o Python/pyfpe.o Python/pyhash.o Python/pylifecycle.o Python/pymath.o Python/pystate.o Python/pythonrun.o Python/pytime.o Python/bootstrap_hash.o Python/structmember.o Python/symtable.o Python/sysmodule.o Python/thread.o Python/traceback.o Python/getopt.o Python/pystrcmp.o Python/pystrtod.o Python/pystrhex.o Python/dtoa.o Python/formatter_unicode.o Python/fileutils.o Python/dynload_shlib.o Python/pydtrace.o Modules/config.o Modules/getpath.o Modules/main.o Modules/gcmodule.o Modules/posixmodule.o Modules/errnomodule.o Modules/pwdmodule.o Modules/_sre.o Modules/_codecsmodule.o Modules/_weakref.o Modules/_functoolsmodule.o Modules/_operator.o Modules/_collectionsmodule.o Modules/_abc.o Modules/itertoolsmodule.o Modules/atexitmodule.o Modules/signalmodule.o Modules/_stat.o Modules/timemodule.o Modules/_threadmodule.o Modules/_localemodule.o Modules/_iomodule.o Modules/iobase.o Modules/fileio.o Modules/bytesio.o Modules/bufferedio.o Modules/textio.o Modules/stringio.o Modules/faulthandler.o Modules/_tracemalloc.o Modules/hashtable.o Modules/symtablemodule.o Modules/xxsubtype.o Python/frozen.o -lcrypt -lpthread -ldl -lutil -lm -lm getbuildinfo.s: Assembler messages: getbuildinfo.s:87427: Error: redefined symbol cannot be used on reloc getbuildinfo.s:261948: Error: redefined symbol cannot be used on reloc lto-wrapper: fatal error: gcc returned 1 exit status compilation terminated. /usr/bin/ld: error: lto-wrapper failed collect2: error: ld returned 1 exit status Oh wait, it created a file called getbuildinfo.s (3,965,403 lines): I attach it to the issue.
Even when compressed with bzip2 -9, the assembly file is still 6.9 MB. Bugzilla doesn't allow files larger than 1 MB. Here is the file: http://haypo.alwaysdata.net/tmp/getbuildinfo.s.bz2
<mock-chroot> sh-5.0# as -a64 -mpower8 -many -mlittle -o test.o getbuildinfo.s getbuildinfo.s: Assembler messages: getbuildinfo.s:87427: Error: redefined symbol cannot be used on reloc getbuildinfo.s:261948: Error: redefined symbol cannot be used on reloc ... addi 5,5,.LC715@toc@l li 4,0 bl _PyObject_AssertFailed.localalias <===== LINE 87427 .LVL7500: .loc 31 2180 9 is_stmt 0 view .LVU23717 ... ... addi 5,5,.LC715@toc@l li 4,0 bl _PyObject_AssertFailed.localalias <===== LINE 261948 .LVL25059: ... --- I found "_PyObject_AssertFailed.localalias" somewhere else: ... .LFE6858: .size _PyObject_AssertFailed,.-_PyObject_AssertFailed .set _PyObject_AssertFailed.localalias,_PyObject_AssertFailed .set _PyObject_AssertFailed.localalias,_PyObject_AssertFailed .align 2 .globl PyInit_xxsubtype .type PyInit_xxsubtype, @function PyInit_xxsubtype: ... --- In Python, _PyObject_AssertFailed() is declared in Include/cpython/object.h by: PyAPI_FUNC(void) _PyObject_AssertFailed( PyObject *obj, const char *expr, const char *msg, const char *file, int line, const char *function); Using gcc -E, I got: # 446 "../Include/cpython/object.h" __attribute__ ((visibility ("default"))) void _PyObject_AssertFailed( PyObject *obj, const char *expr, const char *msg, const char *file, int line, const char *function); And it's implemented as: void _PyObject_AssertFailed(PyObject *obj, const char *expr, const char *msg, const char *file, int line, const char *function) { ... }
The two error locations are the same, namely: bl _PyObject_AssertFailed.localalias Earlier in the asm file, we have: .type _PyObject_AssertFailed, @function _PyObject_AssertFailed: ... .cfi_endproc .LFE6858: .size _PyObject_AssertFailed,.-_PyObject_AssertFailed .set _PyObject_AssertFailed.localalias,_PyObject_AssertFailed .set _PyObject_AssertFailed.localalias,_PyObject_AssertFailed So we're creating that alias twice. If I remove one of them, it assembles fine. Does the source have some typo where it creates the same alias twice?
(In reply to Victor Stinner from comment #7) > > And it's implemented as: > > void > _PyObject_AssertFailed(PyObject *obj, const char *expr, const char *msg, > const char *file, int line, const char *function) > { > ... > } Can you also attach the preprocessed source file that -save-temps also created?
*.localalias aliases are created by symtab_node::noninterposable_alias () method. So, the question is what are the guards that should prevent creating it multiple times (or whether it shouldn't append some number at the end if more than one alias is needed, though I don't really say what would be an advantage to create more than one.
Seems node->call_for_symbol_and_aliases (symtab_node::noninterposable_alias, (void *)&new_node, true); if (new_node) return new_node; should catch that, so the question is why it hasn't been found. Will try to reproduce tomorrow.
Created attachment 47726 [details] preprocessed object.c (gcc -E), bzip2 compressed Here is the gcc -E output of Objects/object.c. Truncated extract (function body replaced with "..."): # 446 "../Include/cpython/object.h" __attribute__ ((visibility ("default"))) void _PyObject_AssertFailed( PyObject *obj, const char *expr, const char *msg, const char *file, int line, const char *function); void _PyObject_AssertFailed(PyObject *obj, const char *expr, const char *msg, const char *file, int line, const char *function) { ... Py_FatalError("_PyObject_AssertFailed"); } I modified the command which creates Objects/object.o to get the preprocessor output, but it's not directly related to the libpython3.9.so linking command (which uses object.o and not object.c).
I used 3.9.0a3 release of Python to reproduce the issue: https://www.python.org/ftp/python/3.9.0/Python-3.9.0a3.tar.xz Here are the full commands to configure Python on ppc64le for Fedora Rawhide with PGO+LTO optimizations. --- export CFLAGS='-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mcpu=power8 -mtune=power8 -fasynchronous-unwind-tables -fstack-clash-protection -D_GNU_SOURCE -fPIC -fwrapv' export CFLAGS_NODIST='-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mcpu=power8 -mtune=power8 -fasynchronous-unwind-tables -fstack-clash-protection -D_GNU_SOURCE -fPIC -fwrapv -fno-semantic-interposition' export CPPFLAGS='' export CXXFLAGS='-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mcpu=power8 -mtune=power8 -fasynchronous-unwind-tables -fstack-clash-protection -D_GNU_SOURCE -fPIC -fwrapv' export FCFLAGS='-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mcpu=power8 -mtune=power8 -fasynchronous-unwind-tables -fstack-clash-protection -I/usr/lib64/gfortran/modules' export FFLAGS='-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mcpu=power8 -mtune=power8 -fasynchronous-unwind-tables -fstack-clash-protection -I/usr/lib64/gfortran/modules' export LDFLAGS='-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -g' export LDFLAGS_NODIST='-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -fno-semantic-interposition -g' export LINKCC=gcc export LT_SYS_LIBRARY_PATH=/usr/lib64: export OPT='-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mcpu=power8 -mtune=power8 -fasynchronous-unwind-tables -fstack-clash-protection -D_GNU_SOURCE -fPIC -fwrapv' ./configure --build=ppc64le-redhat-linux-gnu --host=ppc64le-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --enable-ipv6 --enable-shared --with-computed-gotos=yes --with-dbmliborder=gdbm:ndbm:bdb --with-system-expat --with-system-ffi --enable-loadable-sqlite-extensions --with-dtrace --with-lto --with-ssl-default-suites=openssl --with-valgrind --without-ensurepip --enable-optimizations --- Then run "make". Note: I prefer to "mkdir build; cd build" and run "../configure (...)" to not pollute sources with generated files (and be able to retry from scratch if neeed).
Fedora downstream issue: https://bugzilla.redhat.com/show_bug.cgi?id=1795575
So using the configure setup (minus the -specs=... options) in Comment 13, I was able to recreate the issue using unpatched Python-3.9.0a3 and GCC10. The failing command is: /home/bergner/gcc/install/gcc-fsf-mainline-base/libexec/gcc/powerpc64le-linux/10.0.1/lto1 -quiet -dumpdir ./ -dumpbase libpython3.9.a -mcpu=power8 -mtune=power8 -m64 -auxbase-strip libpython3.9.so.lto.o -g -O2 -version -fno-openmp -fno-openacc -fPIC -fuse-linker-plugin -ffat-lto-objects -flto-partition=none -fno-semantic-interposition -flinker-output=dyn @/tmp/ccikqTcI -o getbuildinfo.s I notice if I drop the -fno-semantic-interposition option, I only get 1: .set _PyObject_AssertFailed.localalias,_PyObject_AssertFailed instead of 2.
It is possible to fix this in the assembler too, but I'm reluctant to do so. If we make some sort of promise that <some referenc to x> .set x,y <maybe lots of lines but no change in y> .set x,y <some reference to x> gives the same results as when only the first .set is present, then gas needs to correctly calculate "no change in y". That is complicated in the general case of an arbitrary expression for y. It's even surprisingly complicated if y is just a single symbol.
So, it seems #0 symtab_node::noninterposable_alias (this=<optimized out>) at ../../gcc/symtab.c:1878 #1 0x0000000010f6e5a0 in function_and_variable_visibility (whole_program=<optimized out>) at ../../gcc/ipa-visibility.c:772 #2 0x00000000111d1ff4 in (anonymous namespace)::whole_program_function_and_variable_visibility () at ../../gcc/ipa-visibility.c:969 #3 (anonymous namespace)::pass_ipa_whole_program_visibility::execute (this=<optimized out>) at ../../gcc/ipa-visibility.c:969 #4 0x00000000109f38c4 in execute_one_pass (pass=pass@entry=0x11896850) at ../../gcc/passes.c:2500 #5 0x00000000109f3578 in execute_ipa_pass_list (pass=0x11896850) at ../../gcc/passes.c:2927 #6 0x0000000011057524 in ipa_passes () at ../../gcc/cgraphunit.c:2660 #7 symbol_table::compile (this=0x7ffff5a30000) at ../../gcc/cgraphunit.c:2737 #8 0x000000001102c318 in lto_main () at ../../gcc/lto/lto.c:658 #9 0x00000000110e7838 in compile_file () at ../../gcc/toplev.c:459 #10 0x000000001073b6fc in do_compile () at ../../gcc/toplev.c:2274 #11 toplev::main (this=0x7fffffffdd76, argc=<optimized out>, argv=<optimized out>) at ../../gcc/toplev.c:2413 #12 0x000000001073de34 in main (argc=<optimized out>, argv=0x7fffffffe1a8) at ../../gcc/main.c:39 is called just once in lto1 for the _PyObject_AssertFailed node, so my guess is that it has been previously created in cc1plus and streamed out and in. The reason why node->call_for_symbol_and_aliases (symtab_node::noninterposable_alias, (void *)&new_node, true); doesn't find the already existing alias is that while _PyObject_AssertFailed and _PyObject_AssertFailed.localalias have the same TREE_TYPE, DECL_CONTEXT and DECL_ATTRIBUTES, the _PyObject_AssertFailed node has TREE_THIS_VOLATILE set on it, while the alias doesn't, so flags_from_type_or_decl (node->decl) returns 0, while flags_from_type_or_decl (fn->decl) returns ECF_NORETURN. During symtab_node::noninterposable_alias (void) it uses copy_node and so should copy even TREE_THIS_VOLATILE, so my guess is that the function isn't marked noreturn explicitly, but it was later on that ipa-pure-const or whatever determined the function is noreturn, and didn't update the aliases (I guess such updating would be dangerous).
Perhaps if we want to really make sure all the flags are what we expect them, we should call clone_function_name_numbered instead of clone_function_name, though I'm afraid I have no idea how well that works across LTO, because I see nothing in the LTO dumping or restoring that would try to maintain clone_fn_ids hash_map. Or we could for each name we try look it up among the aliases and if we find such an alias, retry with another number or punt after trying a few.
Perhaps a workaround (and probably the right thing in any case) would be to add _Py_NO_RETURN to _PyObject_AssertFailed declaration in Include/cpython/object.h because it will always call Py_FatalError which is already _Py_NO_RETURN marked.
I've tried to create small testcase for this: __attribute__((noipa, noreturn)) void foo (void) { while (1) ; } __attribute__((noinline)) void bar (void) { asm (""); foo (); } void baz (int x) { if (x) bar (); } and extern void bar (void); void qux (int x) { if (!x) bar (); } ./xgcc -B ./ -O2 -flto -ffat-lto-objects pr93384_0.c -S -fpic -fno-semantic-interposition ./xgcc -B ./ -O2 -flto -ffat-lto-objects pr93384_1.c -S -fpic -fno-semantic-interposition as -o pr93384_0.{o,s} as -o pr93384_1.{o,s} ./xgcc -B ./ -O2 -flto -shared -fpic -fno-semantic-interposition -o pr93384.so pr93384_{0,1}.o -save-temps -v -flto-partition=one but unfortunately (at least on x86_64-linux) it doesn't reproduce this, the bar.localalias gets suffixed in one case to bar.localalias.lto_priv.0 and in another one (though it didn't exist in the assembly at that point) to bar.localalias.lto_priv.1.
(In reply to Jakub Jelinek from comment #20) > I've tried to create small testcase for this: > __attribute__((noipa, noreturn)) void foo (void) { while (1) ; } > __attribute__((noinline)) void bar (void) { asm (""); foo (); } > void baz (int x) { if (x) bar (); } > and > extern void bar (void); > void qux (int x) { if (!x) bar (); } > > ./xgcc -B ./ -O2 -flto -ffat-lto-objects pr93384_0.c -S -fpic > -fno-semantic-interposition > ./xgcc -B ./ -O2 -flto -ffat-lto-objects pr93384_1.c -S -fpic > -fno-semantic-interposition > as -o pr93384_0.{o,s} > as -o pr93384_1.{o,s} > ./xgcc -B ./ -O2 -flto -shared -fpic -fno-semantic-interposition -o > pr93384.so pr93384_{0,1}.o -save-temps -v -flto-partition=one > > but unfortunately (at least on x86_64-linux) it doesn't reproduce this, the > bar.localalias gets suffixed in one case to bar.localalias.lto_priv.0 and in > another one (though it didn't exist in the assembly at that point) to > bar.localalias.lto_priv.1. Ah, what matters is -flto-partition=none that I was missing from the above. With that the above reproduces it: ./xgcc -B ./ -O2 -flto -ffat-lto-objects pr93384_0.c -c -fpic \ -fno-semantic-interposition ./xgcc -B ./ -O2 -flto -ffat-lto-objects pr93384_1.c -c -fpic \ -fno-semantic-interposition ./xgcc -B ./ -O2 -flto -shared -fpic -fno-semantic-interposition -o \ pr93384.so pr93384_{0,1}.o -save-temps -v -flto-partition=none grep localalias pr93384_0.s .set bar.localalias,bar .set bar.localalias,bar jmp bar.localalias call bar.localalias Now, gas on x86_64-linux can deal with that, but on powerpc64le-linux can't.
To be precise, the above testcase doesn't reproduce the ppc64le assembler error, for that to reproduce it is needed that there is at least one direct call to the *.localalias emitted before the two .set directives and then at least one another direct call to it, while the above testcase has the two .set directives first and then the two calls. But, still, we shouldn't emit more than one alias with the same name, even if it has the same target.
Ah, just small change is needed to make it reproduce the problem: $ cat pr93384_0.c void bar (void); __attribute__((noipa)) void quux (int x) { if (x == 5) bar (); } __attribute__((noipa, noreturn)) void foo (void) { while (1) ; } __attribute__((noinline)) void bar (void) { asm (""); quux (7); foo (); } void baz (int x) { if (x) bar (); } $ cat pr93384_1.c extern void bar (void); void qux (int x) { if (!x) bar (); } $ gcc -O2 -flto -ffat-lto-objects pr93384_0.c -c -fpic -fno-semantic-interposition $ gcc -O2 -flto -ffat-lto-objects pr93384_1.c -c -fpic -fno-semantic-interposition $ gcc -O2 -flto -shared -fpic -fno-semantic-interposition -o pr93384.so pr93384_{0,1}.o -flto-partition=none /tmp/ccbfJD5V.s: Assembler messages: /tmp/ccbfJD5V.s:33: Error: redefined symbol cannot be used on reloc lto-wrapper: fatal error: gcc returned 1 exit status compilation terminated. /usr/bin/ld: error: lto-wrapper failed collect2: error: ld returned 1 exit status
Reading through the PR I understand it as follows 1) we create localalias for a symbol 2) ipa-pure-const founds that the function is noreturn. Here seems to be a bug, since it does TREE_THIS_VOLATILE (current_function_decl) = 1 while what it should really do is to walk symbol and all its clones and if the availability is AVAILABLE set the flag. It should also update associated thunks. This is implemented for const/pure in set_const_flag. I will look into this. 3) later we introduce new refernece and want local alias again. create_localalias no longer sees the original local alias as one (because it is now different) and creates new one with identical assembler name 3 is a bug which is not solved by 2 and I think it is a fallout of the reproducible clone names patch we added few releases back. We used to be creating .localalias.<xyz> while now we stop the ID. Where we lose the clone ID? I am not sure how hard would be clone ID right (i.e. if we just forget to stream it or it happens because of symbol merging), Alternative would be to modify clone_function_name_1 to lookup assembler name in the symbol table and in the case of clash increment the clone ID. We do not produce very many clones of a single symbol name so this could work well enough.
Yeah, I think that is a good summary. I wasn't sure if 2) is safe, but perhaps it is if it comes from analysis of the function body (for user attributes, I believe I've seen (or even wrote, don't remember) an alias on error(3) function, where error itself can't be noreturn, but the alias would be something we always call with arguments that will make it noreturn, and that is something we don't want to propagate in between the decls). And yes, for 3) that is something I've mentioned above, though not in the form of looking up the assembler name, but walking the aliases, though I guess looking up the assembler name is better. Dunno if we can punt if we don't find a suitable name in a certain number of attempts, I see most of the callers that can cope with the function returning NULL, but at least one spot asserts that it returns non-NULL.
Created attachment 47741 [details] gcc10-pr93384.patch Untested fix (only the make localalias name unique part, not the ipa-pure-const.c change).
The master branch has been updated by Jakub Jelinek <jakub@gcc.gnu.org>: https://gcc.gnu.org/g:5fb07870fa4c86f529930bae76689ed5bdfcb192 commit r10-6359-g5fb07870fa4c86f529930bae76689ed5bdfcb192 Author: Jakub Jelinek <jakub@redhat.com> Date: Thu Jan 30 21:32:36 2020 +0100 cgraph: Avoid creating multiple *.localalias aliases with the same name [PR93384] The following testcase FAILs on powerpc64le-linux with assembler errors, as we emit a call to bar.localalias, then .set bar.localalias, bar twice and then another call to bar.localalias. The problem is that bar.localalias can be created at various stages and e.g. ipa-pure-const can slightly adjust the original decl, so that the existing bar.localalias isn't considered usable (different flags_from_decl_or_type). In that case, we'd create another bar.localalias, which clashes with the existing name. Fixed by retrying with another name if it is already present. The various localalias aliases shouldn't be that many, from different partitions they would be lto_priv suffixed and in most cases they would already have the same type/flags/attributes. 2020-01-30 Jakub Jelinek <jakub@redhat.com> PR lto/93384 * symtab.c (symtab_node::noninterposable_alias): If localalias already exists, but is not usable, append numbers after it until a unique name is found. Formatting fix. * gcc.dg/lto/pr93384_0.c: New test. * gcc.dg/lto/pr93384_1.c: New file.
Fixed on the trunk.
Jakub Jelinek: "Fixed on the trunk." Oh wow, that was quick! Thanks for the fix!