[PATCH] Try LTO partial linking. (Was: Speed of compiling gimple-match.c)

[Giuliano Belinassi]

Hi, all.

Please CC me when I am mentioned into a mail.

On Thu, 2021-05-20 at 15:16 +0200, Richard Biener via Gcc wrote:
> On Thu, May 20, 2021 at 3:06 PM Martin Liška <mliska@suse.cz> wrote:
> > 
> > On 5/20/21 2:54 PM, Richard Biener wrote:
> > > So why did you go from applying this per-file to multiple files?
> > 
> > When I did per-file for {gimple,generic}-match.c I hit the
> > following issue with lto.priv symbols:
> > 
> > /usr/lib64/gcc/x86_64-suse-linux/10/../../../../x86_64-suse-
> > linux/bin/ld: error: libbackend.a(generic-match.o): multiple
> > definition of 'wi::to_wide(tree_node const*) [clone .part.0] [clone
> > .lto_priv.0]'
> > /usr/lib64/gcc/x86_64-suse-linux/10/../../../../x86_64-suse-
> > linux/bin/ld: libbackend.a(gimple-match.o): previous definition
> > here
> > /usr/lib64/gcc/x86_64-suse-linux/10/../../../../x86_64-suse-
> > linux/bin/ld: error: libbackend.a(generic-match.o): multiple
> > definition of 'TYPE_VECTOR_SUBPARTS(tree_node const*) [clone
> > .part.0] [clone .lto_priv.0]'
> > /usr/lib64/gcc/x86_64-suse-linux/10/../../../../x86_64-suse-
> > linux/bin/ld: libbackend.a(gimple-match.o): previous definition
> > here
> > /usr/lib64/gcc/x86_64-suse-linux/10/../../../../x86_64-suse-
> > linux/bin/ld: error: libbackend.a(generic-match.o): multiple
> > definition of 'vec<constructor_elt, va_gc,
> > vl_embed>::operator[](unsigned int) [clone .part.0] [clone
> > .lto_priv.0]'
> > 
> > Any idea what was I doing wrong?
> 
> Nothing in particular I think - you're just hitting the issue that
> LTO
> produces new symbols and that those
> can obviously clash.  Giuliano hit the very same issue.  When not
> doing partial links those internal
> symbols pose no problem, but with -r -flinker-output=nolto-rel and
> re-linking the produced objects
> they obviously do.  ELF has no solution for this though, but I think
> we could strip those from the
> partially linked object - if WPA would give us a list of objects the
> link step could postprocess
> the object with objcopy or maybe a custom linker script could do the
> trick as well.

I've "fixed" this issue in my branch by mangling any promoted to public
symbol. I've also disabled the "ipa-split" pass in the paper branch
because of some created symbols which I was not able to fix in time.
Perhaps this goes away if you disable it.

Perhaps we should work on getting the autopar branch merged into trunk.
There are several issues which must be fixed and I don't think it will
be ready for this next release. The main ones that I remember from the
top of my head:

1- Fix the driver to use SPEC language for the multiple required calls
to `as`, instead of injecting code for that directly on the `void
execute()` function.

2- Merge my custom partitioner for using the default LTO partitioner.
The default LTO partitioner were hitting the assertions about COMDAT
being split into multiple partitions.

3- Fix the issue with the ipa-split pass.

Perhaps we should further explore avoiding partial linking altogether
and concat the assembler files.

Thank you,
Giuliano.

> 
> So your workaround is to only ever have a single LTO produced object
> file participating in the
> final links ;)
> 
> Richard.
> 
> > 
> > Martin