How to debug while using LTO?

[Stefan Schulze Frielinghaus]

On Thu, Nov 24, 2022 at 05:53:53PM +0100, Richard Biener wrote:
> 
> 
> > Am 24.11.2022 um 17:28 schrieb Stefan Schulze Frielinghaus via Gcc <gcc@gcc.gnu.org>:
> > 
> > Hi everyone,
> > 
> > Currently I'm looking into a wrong-code bug and would like to understand
> > a certain optimization done by combine during local transformation.
> > Without LTO I would simply debug cc1 and step through combine.  However,
> > with LTO enabled AFAIK I have to debug lto1 instead.  In order to get
> > the lto1 command line of interest according to
> > https://gcc.gnu.org/legacy-ml/gcc/2009-11/msg00047.html
> > I have to pass -Wl,-debug to gcc in order to get the command for
> > collect2 to which itself I have to pass -plugin-opt=-debug in order to
> > get the command for lto-wrapper.  According to the aforementioned mail I
> > should add option -debug to lto-wrapper, however, it appears to me that
> > option -debug was removed.  I gave options -v and -### a chance without
> > luck, i.e., those only print the usual environment variables and
> > afterwards a list of object files like
> > 
> > /tmp/ccPEIV35.ltrans0.ltrans.o
> > /tmp/ccNmpKfS.debug.temp.o
> > /tmp/cceiCIFg.debug.temp.o
> > /tmp/ccZ4Qc7E.debug.temp.o
> > ...
> > 
> > but no lto1 command.  Thus, how do you retrieve the lto1 command?
> > 
> > While desperate I retrieved it manually via strace.  However, the lto1
> > command refers to temporary files which have been erased meanwhile.  I
> > actually didn't expect that because I added -save-temps to all the
> > intermediate commands which is also reflected in the environment
> > variable COLLECT_GCC_OPTIONS.  Thus, how do you keep temporary files?
> 
> Adding -v -save-temps and then running gdb on the lto1 command works and is what I usually do.

Strangely this is not the case for the current project I'm looking at.  The
link step is done via

/usr/local/bin/s390x-linux-gnu-gcc -save-temps \
  -O3 -DNDEBUG -flto=auto -fno-fat-lto-objects \
  -march=z15 -static -nostdlib \
  -Wl,--fatal-warnings -Wl,--no-warn-rwx-segments \
  -Werror -Wall -Wundef -T/devel/foo.ld \
  obj1.o ... objN.o

where -save-temps is given and was also given during compile time of each
object obj1.o ... objN.o file.  Anyhow, adding -Wl,-debug to the link command
above I see that -save-temps is also included in the environment variable
COLLECT_GCC_OPTIONS="... -save-temps ..." and I get the linker command

/usr/local/lib/gcc/s390x-linux-gnu/13.0.0/../../../../s390x-linux-gnu/bin/ld \
  -plugin /usr/local/libexec/gcc/s390x-linux-gnu/13.0.0/liblto_plugin.so \
  -plugin-opt=/usr/local/libexec/gcc/s390x-linux-gnu/13.0.0/lto-wrapper \
  -plugin-opt=-fresolution=foo.res \
  -m elf64_s390 -static -o foo \
  -L/usr/local/lib/gcc/s390x-linux-gnu/13.0.0 \
  -L/usr/local/lib/gcc/s390x-linux-gnu/13.0.0/../../../../s390x-linux-gnu/lib \
  --fatal-warnings --no-warn-rwx-segments \
  obj1.o ... objN.o -lgcc -T /devel/foo.ld

to which I add -plugin-opt=-debug in order to get the lto-wrapper command

/usr/local/libexec/gcc/s390x-linux-gnu/13.0.0/lto-wrapper \
  -fresolution=foo.res -flinker-output=exec \
  obj1.o ... objN.o

to which I add -v -save-temps in order to actually get the lto1 command.  From
the output printed to stderr I see one lto1 command:

/usr/local/libexec/gcc/s390x-linux-gnu/13.0.0/lto1 -quiet -dumpbase ./a.wpa \
  -march=z15 -m64 -mzarch -g -O3 -version -fno-openmp -fno-openacc -fno-pie \
  -fcf-protection=none -fltrans-output-list=/tmp/ccsuw4Nj.ltrans.out \
  -fwpa=32 -fresolution=foo.res -flinker-output=exec @./a.wpa.args.0

However, this lto1 invocation does no local transformation.  Having a look at
the strace output I see a second lto1 invocation:

/usr/local/libexec/gcc/s390x-linux-gnu/13.0.0/lto1 -quiet \
  -dumpbase ./a.ltrans0.ltrans -march=z15 -m64 -mzarch -g -O3 \
  -fno-openmp -fno-openacc -fno-pie -fcf-protection=none \
  -fltrans /tmp/ccsuw4Nj.ltrans0.o -o /tmp/cc69wMp3.s

This lto1 invocation seems to originate from Makefile /tmp/cc5piuXT.mk
generated by lto-wrapper.  To my surprise the Makefile as well as the temporary
file /tmp/ccsuw4Nj.ltrans0.o are removed although -save-temps was given all the
time.  Stepping through lto-wrapper with gdb I see that

  save_temps = 1;

is never executed while walking over decoded_options.

For the time being I hard coded save_temps=1 into lto-wrapper.cc and I am now
able to debug the lto1 invocation of interest.  Not sure whether there
is a more elegant way?

All this is based upon a rather huge CMake based project and so far I didn't
find the time to come up with a small reproducer.  Still hoping this might be
useful for someone else.

Cheers,
Stefan