[Bug tree-optimization/78529] [7 Regression] gcc.c-torture/execute/builtins/strcat-chk.c failed with lto/O2

["wilson at gcc dot gnu.org" <gcc-bugzilla at gcc dot gnu dot org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=78529

--- Comment #26 from Jim Wilson <wilson at gcc dot gnu.org> ---
I can reproduce the problem with this new reduced testcase.  I don't have
knowledge of all of the details how the gcc implementation of LTO works, but my
understanding goes something like this.

The testcase is defining memset.  Memset is a reserved identifier (ISO C
section 7.1.3).  So the testcase is violating the ISO C standard.

The memset definition in the input source is LTO optimized, which means it gets
turned into an LTO symbol that is not immediately visible to the linker.  The
linker sees the startfiles first, and then the LTO optimized files.  The crt0.o
file has a reference to memset.  The linker can't use the LTO optimized memset
to satisfy this reference, because it is still a special LTO symbol not a
normal symbol.  So it pulls in memset from the C library to satisfy the
reference.  The linker then looks at the LTO optimized files, calls the
compiler to convert them from LTO symbols into normal symbols, and then
discovers that it has two memset functions, and because
--allow-multiple-definitions is given, it arbitrarily discards one instead of
giving an error.  Since the memset from libc.a was already linked in, I don't
think it has much choice, and has to drop the new one from LTO.

The main function was LTO optimized expecting that it would call the LTO
optimized memset function, which uses a restricted register set, thus allowing
main to use normally call clobbered registers across the memset call.  However,
if the linker drops the LTO optimized memset, then we get a call to the library
memset, which clobbers all call clobbered regs, and the main function fails.

The problem is primarily with the testcase, and secondarily with how LTO works.

If the LTO link converted the LTO symbols into normal symbols before trying to
resolve unsatisfied symbols from crt0.o, that would appear to solve the
problem.  This would require changes to the linker, changes that may or may not
be reasonable.

If crt0.o didn't call memset, that would solve the problem, but that isn't a
very reasonable solution.  There may also be issues with other standard library
functions and/or other start files.

If the newlib C library was built as a shared library that would appear to
solve the problem, as the LTO symbol would override the shared library symbol,
but that isn't a reasonable solution for an embedded target.

The easy solution is to stop running this test on embedded targets that don't
have a shared C library.