[RFC][4.3 regression] Fix massive compile-time and memory consumption for large numbers of SFTs

["Diego Novillo" <dnovillo at google dot com>]

I have a C++ test case that can be compiled in ~2m on gcc 4.2, but it
takes more than 25 minutes and 3Gb of RAM to compile on gcc 4.3.  The
culprit is the creation of SFTs.  This program has more 1,300
structure variables for which SFTs can be created, and when we create
more than 2 or 3 SFTs per variable, we get into the multi-gigabyte
memory consumption problem.

To fix it, I simply added a new --param definition that blocks
create_structure_vars from running if the number of partitionable
symbols is beyond a certain threshold.  The heuristic is very simple
and it cuts down compilation time back to 100 seconds.

Since we are not going to support SFTs anymore, this patch only makes
sense for the 4.3 branch.  I did not want to do anything fancier than
just counting the number of candidate variables, because I really saw
no point to it.  So, the idea is that if there are more than 800
symbols, we give up early and not bother creating SFTs.

I have been trying to find a bugzilla PR that resembles this, but I
haven't run into anything obvious.  The testcase I have is pretty
horrible (about 5Mb), so it's not really something I'd like to put in
the testsuite.

To summarize, the usefulness of the heuristic is limited to pre 4.4
compilers.  The checking is very simplistic on purpose, if there are
too many symbols, punt on the expensive SFT creation.  The question I
have for the branch maintainers is whether this is acceptable.  There
are no risks to correctness, all the patch does is disable SFT
creation which only affects optimization opportunities.
Unfortunately, I don't seem to have a good test case to go with it.

It has been bootstrapped and tested on i686 and x86_64.  OK for 4.3 branch?


Thanks.  Diego.

Attachment: 20080715-add-cost-heuristic-to-SFT.diff.txt
Description: Text document