Improve partitioning decisions
H.J. Lu
hjl.tools@gmail.com
Mon Apr 30 07:50:00 GMT 2018
On Sat, Apr 28, 2018 at 11:55 AM, Jan Hubicka <hubicka@ucw.cz> wrote:
> Hi,
> with number of cores increasing our partitioning algorithm needs some love,
> because it works quite bad.
>
> This patch fixes few issues with ballanced partitioning algorithm:
> 1) I have switched all size/boundary summaries to 64bit arthmetics as they
> may overflow for very large units
> 2) internal calls within partition was misaccounted
> 3) I no longer account external references (as those will never be local
> no matter how we partition)
> 4) the range where we search for best split is not +- 1/8 of partition.
> 5) we no longer look for worst possible split, but for best one.
> 6) I have added bit extra debug info.
>
> I have checked this with Firefox build. Previously we did 31 instead of 32 partitions
> and the difference between largest and smallest was 170%. Now we do 32 partitions and get
> the difference is "only" about 70%. The resulting code quality improves a bit and
> we get 260s for largest partition build time instead of 305s.
>
> I will commit it into the mainline and backport to release branches after some soaking
> (and 8.1 release), since without the patch partitioning decisions with larger # of partitions
> are really pretty bad.
>
> Situation is still not perfect - I do not get any speedup for increasing partitions to 64
> becuase we still get too much of extra streaming. Something to track this stage1.
>
> Honza
>
> * lto-partition.c: Include sreal.h
> (add_symbol_to_partition_1): Use size instead of self_size
> for size estimate.
> (account_reference_p): New.
> (lto_balanced_map): Use 64bit arithmetics for size calculatoins; cleanup;
> fix accounting errors in boundary size; add debug output; combine cost
> as cost/size instead of cost/internal; reduce the partitioning error to
> +- 1/8 of the parttion size.
This caused:
FAIL: gcc.dg/lto/20081204-2 c_lto_20081204-2_0.o-c_lto_20081204-2_0.o
link, -w -flto -fPIC -r -nostdlib (internal compiler error)
FAIL: gcc.dg/lto/20090914-2 c_lto_20090914-2_0.o-c_lto_20090914-2_0.o
link, -O2 -flto -fuse-linker-plugin (internal compiler error)
FAIL: gcc.dg/lto/20091014-1 c_lto_20091014-1_0.o-c_lto_20091014-1_0.o
link, -fPIC -r -nostdlib -flto (internal compiler error)
FAIL: gcc.dg/lto/20100603-1 c_lto_20100603-1_0.o-c_lto_20100603-1_0.o
link, -O0 -flto -fuse-linker-plugin -fno-fat-lto-objects (internal
compiler error)
FAIL: gcc.dg/lto/20100603-1 c_lto_20100603-1_0.o-c_lto_20100603-1_0.o
link, -O2 -flto -fuse-linker-plugin (internal compiler error)
FAIL: g++.dg/lto/20081120-1
cp_lto_20081120-1_0.o-cp_lto_20081120-1_1.o link, -flto -r -nostdlib
(internal compiler error)
FAIL: g++.dg/lto/20081120-2
cp_lto_20081120-2_0.o-cp_lto_20081120-2_1.o link, -flto -r -nostdlib
(internal compiler error)
on x86.
--
H.J.
More information about the Gcc-patches
mailing list