Re: compiling very large functions

[Bradley Lucier <lucier at math dot purdue dot edu>]

The gcc developers have been very cooperative over the years in  
working to solve problems that I've had in compiling large machine- 
generated programs.  For example, when gcse was disabled for large  
flow flow graphs in 2000, the warn_disabled_optimization flag was  
added at my suggestion.  As Steven Bosscher hinted at, however, it  
appears that the disabled optimization warning has not been used at  
all beyond its introduction:

% grep -R disabled_optimization *
gcc/.svn/text-base/ChangeLog-2000.svn-base:     * toplev.c  
(warn_disabled_optimization): Declare new warning flag.
gcc/.svn/text-base/ChangeLog-2000.svn-base:     * flags.h  
(warn_disabled_optimization): Add it here.
gcc/.svn/text-base/common.opt.svn-base:Common Var 
(warn_disabled_optimization)
gcc/.svn/text-base/gcse.c.svn-base:      warning  
(OPT_Wdisabled_optimization,
gcc/.svn/text-base/gcse.c.svn-base:      warning  
(OPT_Wdisabled_optimization,
gcc/ChangeLog-2000:     * toplev.c (warn_disabled_optimization):  
Declare new warning flag.
gcc/ChangeLog-2000:     * flags.h (warn_disabled_optimization): Add  
it here.
gcc/common.opt:Common Var(warn_disabled_optimization)
gcc/gcse.c:      warning (OPT_Wdisabled_optimization,
gcc/gcse.c:      warning (OPT_Wdisabled_optimization,

A grep of 'PARAM_VALUE.*MAX' may give a more accurate idea of where  
optimization passes have been throttled for various reasons.

In reporting runtime problems, my experience has been that if there  
is one specific pass whose runtime overwhelms the runtime of all the  
other passes, then people respond quickly to deal with it.  I've  
tended to use a relatively low level of optimization in my programs (- 
O1 -fschedule-insns2 -fno-math-errno -fno-trapping-math -fwrapv -fno- 
strict-aliasing) and even as new tree-ssa passes were added to -O1,  
the runtime and space requirements were kept reasonable (perhaps  
after a bit of work).

Now I'm finding that more aggressive optimizations can significantly  
speed up these codes; I'm also finding that these newly-attempted  
optimization passes take up so much memory (especially on darwin G5  
for some reason) that they can't be applied except on very large  
machines.  (One of my codes took only six minutes, but over 10.1 GB  
of memory, to compile; smaller examples are given in PR 29374.)   
Perhaps these passes intrinsically require large amounts of memory,  
or perhaps the algorithms and data structures used have not yet been  
examined critically for very large programs.  When I find time I plan  
to discover which specific optimizations require large memory;  
perhaps with this data the passes involved can be studied more  
closely to see whether the memory requirements can be cut.

With that background, I'd like to request that optimization passes  
not be throttled back by default, especially in stage 1 of a release  
cycle.  I fear that this would unduly hide problems that might be  
solved with a reasonable amount of effort.  It's not clear to me how  
many new optimization passes and data structures have been stress- 
tested on very large programs; we may find that most problems that  
appear now can be fixed if they are examined in isolation with the  
right input data.

Brad