Re: A recent patch decreased GCC's memory consumption.

[Jan Hubicka <jh at suse dot cz>]

Paolo, Zdenek,
can your patch be reason for the 17% memory consumption decrease for
memory outside GGC area reported in insn-attrtab?
I am asking because I saw similar decrease in my prevoius patch to
variable annotations that should've monotonously increase memory
consumption similar as this one.  I would probably need to investigate
if some of other patches don't show itself to be "guilty" :)

Honza
> Hi,
> 
> I am a friendly script caring about memory consumption in GCC.  Please
> contact jh@suse.cz if something is going wrong.
> 
> Comparing memory consumption on compilation of combine.i, insn-attrtab.i,
> and generate-3.4.ii I got:
> 
> 
> comparing empty function compilation at -O0 level:
>     Overall memory needed: 18297k -> 18305k
>     Peak memory use before GGC: 2235k
>     Peak memory use after GGC: 1942k
>     Maximum of released memory in single GGC run: 293k
>     Garbage: 423k
>     Leak: 2273k
>     Overhead: 446k
>     GGC runs: 3
> 
> comparing empty function compilation at -O0 -g level:
>     Overall memory needed: 18313k -> 18321k
>     Peak memory use before GGC: 2263k
>     Peak memory use after GGC: 1970k
>     Maximum of released memory in single GGC run: 293k
>     Garbage: 426k
>     Leak: 2305k
>     Overhead: 450k
>     GGC runs: 3
> 
> comparing empty function compilation at -O1 level:
>     Overall memory needed: 18397k -> 18405k
>     Peak memory use before GGC: 2235k
>     Peak memory use after GGC: 1942k
>     Maximum of released memory in single GGC run: 293k
>     Garbage: 427k
>     Leak: 2275k
>     Overhead: 446k
>     GGC runs: 4
> 
> comparing empty function compilation at -O2 level:
>     Overall memory needed: 18409k -> 18417k
>     Peak memory use before GGC: 2236k
>     Peak memory use after GGC: 1942k
>     Maximum of released memory in single GGC run: 294k
>     Garbage: 430k -> 430k
>     Leak: 2275k
>     Overhead: 447k -> 447k
>     GGC runs: 4
> 
> comparing empty function compilation at -O3 level:
>     Overall memory needed: 18409k -> 18417k
>     Peak memory use before GGC: 2236k
>     Peak memory use after GGC: 1942k
>     Maximum of released memory in single GGC run: 294k
>     Garbage: 430k -> 430k
>     Leak: 2275k
>     Overhead: 447k -> 447k
>     GGC runs: 4
> 
> comparing combine.c compilation at -O0 level:
>     Overall memory needed: 28465k -> 28473k
>     Peak memory use before GGC: 9288k
>     Peak memory use after GGC: 8803k
>     Maximum of released memory in single GGC run: 2642k
>     Garbage: 37567k
>     Leak: 6454k
>     Overhead: 4875k
>     GGC runs: 281
> 
> comparing combine.c compilation at -O0 -g level:
>     Overall memory needed: 30541k -> 30549k
>     Peak memory use before GGC: 10834k
>     Peak memory use after GGC: 10463k
>     Maximum of released memory in single GGC run: 2320k
>     Garbage: 38128k
>     Leak: 9346k
>     Overhead: 5576k
>     GGC runs: 272
> 
> comparing combine.c compilation at -O1 level:
>     Overall memory needed: 29474k
>     Peak memory use before GGC: 16963k
>     Peak memory use after GGC: 16792k
>     Maximum of released memory in single GGC run: 2251k
>     Garbage: 55759k -> 55778k
>     Leak: 6489k -> 6481k
>     Overhead: 9966k -> 9963k
>     GGC runs: 357
> 
> comparing combine.c compilation at -O2 level:
>     Overall memory needed: 29474k
>     Peak memory use before GGC: 16967k
>     Peak memory use after GGC: 16792k
>     Maximum of released memory in single GGC run: 2373k -> 2374k
>     Garbage: 71973k -> 71989k
>     Leak: 6602k -> 6604k
>     Overhead: 11954k -> 11953k
>     GGC runs: 412 -> 413
> 
> comparing combine.c compilation at -O3 level:
>     Overall memory needed: 29602k
>     Peak memory use before GGC: 18067k
>     Peak memory use after GGC: 17600k
>     Maximum of released memory in single GGC run: 3689k -> 3692k
>     Garbage: 106221k -> 106245k
>     Leak: 6684k
>     Overhead: 16876k -> 16872k
>     GGC runs: 460
> 
> comparing insn-attrtab.c compilation at -O0 level:
>     Overall memory needed: 89650k
>     Peak memory use before GGC: 71193k
>     Peak memory use after GGC: 44699k
>     Maximum of released memory in single GGC run: 37868k
>     Garbage: 132141k
>     Leak: 9518k
>     Overhead: 16956k
>     GGC runs: 211
> 
> comparing insn-attrtab.c compilation at -O0 -g level:
>     Overall memory needed: 90826k
>     Peak memory use before GGC: 72354k
>     Peak memory use after GGC: 45966k
>     Maximum of released memory in single GGC run: 37868k
>     Garbage: 133548k
>     Leak: 10981k
>     Overhead: 17351k
>     GGC runs: 209
> 
> comparing insn-attrtab.c compilation at -O1 level:
>     Overall memory needed: 93730k
>     Peak memory use before GGC: 71858k
>     Peak memory use after GGC: 67990k
>     Maximum of released memory in single GGC run: 31668k
>     Garbage: 229899k -> 229914k
>     Leak: 9343k
>     Overhead: 29520k -> 29517k
>     GGC runs: 220
> 
> comparing insn-attrtab.c compilation at -O2 level:
>   Ovarall memory allocated via mmap and sbrk decreased from 124006k to 105134k, overall -17.95%
>     Overall memory needed: 124006k -> 105134k
>     Peak memory use before GGC: 79550k
>     Peak memory use after GGC: 73715k
>     Maximum of released memory in single GGC run: 30212k
>     Garbage: 282712k -> 282755k
>     Leak: 9345k
>     Overhead: 35776k -> 35780k
>     GGC runs: 243 -> 244
> 
> comparing insn-attrtab.c compilation at -O3 level:
>   Ovarall memory allocated via mmap and sbrk decreased from 123766k to 105158k, overall -17.70%
>     Overall memory needed: 123766k -> 105158k
>     Peak memory use before GGC: 79576k
>     Peak memory use after GGC: 73740k
>     Maximum of released memory in single GGC run: 30406k -> 30407k
>     Garbage: 283552k -> 283574k
>     Leak: 9350k
>     Overhead: 36009k -> 36008k
>     GGC runs: 247
> 
> comparing Gerald's testcase PR8361 compilation at -O0 level:
>     Overall memory needed: 119066k
>     Peak memory use before GGC: 92331k
>     Peak memory use after GGC: 91417k
>     Maximum of released memory in single GGC run: 19248k
>     Garbage: 212896k
>     Leak: 48116k
>     Overhead: 21264k
>     GGC runs: 417
> 
> comparing Gerald's testcase PR8361 compilation at -O0 -g level:
>     Overall memory needed: 131642k
>     Peak memory use before GGC: 104692k
>     Peak memory use after GGC: 103652k
>     Maximum of released memory in single GGC run: 18936k
>     Garbage: 219492k
>     Leak: 71548k
>     Overhead: 27169k
>     GGC runs: 390
> 
> comparing Gerald's testcase PR8361 compilation at -O1 level:
>     Overall memory needed: 119502k
>     Peak memory use before GGC: 96665k
>     Peak memory use after GGC: 94462k
>     Maximum of released memory in single GGC run: 17940k
>     Garbage: 442217k -> 442205k
>     Leak: 50183k
>     Overhead: 100280k -> 100280k
>     GGC runs: 562
> 
> comparing Gerald's testcase PR8361 compilation at -O2 level:
>     Overall memory needed: 119558k
>     Peak memory use before GGC: 96693k
>     Peak memory use after GGC: 94490k
>     Maximum of released memory in single GGC run: 18081k
>     Garbage: 497568k -> 497579k
>     Leak: 51152k
>     Overhead: 58887k -> 58889k
>     GGC runs: 614
> 
> comparing Gerald's testcase PR8361 compilation at -O3 level:
>     Overall memory needed: 121138k
>     Peak memory use before GGC: 97647k
>     Peak memory use after GGC: 96143k
>     Maximum of released memory in single GGC run: 18476k
>     Garbage: 517902k -> 517913k
>     Leak: 51126k
>     Overhead: 58882k -> 58884k
>     GGC runs: 620
> 
> comparing PR rtl-optimization/28071 testcase compilation at -O0 level:
>     Overall memory needed: 137290k -> 137634k
>     Peak memory use before GGC: 81587k
>     Peak memory use after GGC: 58467k
>     Maximum of released memory in single GGC run: 45166k
>     Garbage: 148528k
>     Leak: 7542k
>     Overhead: 25329k
>     GGC runs: 82
> 
> comparing PR rtl-optimization/28071 testcase compilation at -O0 -g level:
>     Overall memory needed: 138014k
>     Peak memory use before GGC: 82233k
>     Peak memory use after GGC: 59113k
>     Maximum of released memory in single GGC run: 45232k
>     Garbage: 148738k
>     Leak: 9309k
>     Overhead: 25824k
>     GGC runs: 88
> 
> comparing PR rtl-optimization/28071 testcase compilation at -O1 level:
>     Overall memory needed: 421226k -> 421134k
>     Peak memory use before GGC: 199539k
>     Peak memory use after GGC: 193352k
>     Maximum of released memory in single GGC run: 112477k
>     Garbage: 287375k -> 287375k
>     Leak: 29778k
>     Overhead: 32193k -> 32193k
>     GGC runs: 98
> 
> comparing PR rtl-optimization/28071 testcase compilation at -O2 level:
>     Overall memory needed: 343394k -> 343746k
>     Peak memory use before GGC: 199532k
>     Peak memory use after GGC: 193345k
>     Maximum of released memory in single GGC run: 111911k -> 111908k
>     Garbage: 364383k -> 364381k
>     Leak: 30361k
>     Overhead: 47296k -> 47296k
>     GGC runs: 104
> 
> comparing PR rtl-optimization/28071 testcase compilation at -O3 -fno-tree-pre -fno-tree-fre level:
>     Overall memory needed: 749294k -> 749290k
>     Peak memory use before GGC: 317673k
>     Peak memory use after GGC: 296148k
>     Maximum of released memory in single GGC run: 186409k -> 186406k
>     Garbage: 504462k -> 504459k
>     Leak: 45414k
>     Overhead: 60273k -> 60272k
>     GGC runs: 99
> 
> Head of the ChangeLog is:
> 
> --- /usr/src/SpecTests/sandbox-britten-memory/x86_64/mem-result/ChangeLog	2006-12-22 12:13:26.000000000 +0000
> +++ /usr/src/SpecTests/sandbox-britten-memory/gcc/gcc/ChangeLog	2006-12-23 03:55:20.000000000 +0000
> @@ -1,3 +1,68 @@
> +2006-12-23  Jan Hubicka  <jh@suse.cz>
> +
> +	* tree-flow-inline.h (var_ann): External variable annotations are
> +	unshared too.
> +	(tree_common_ann): Handle correctly unshared variables annotations.
> +	* tree-dfa.c (create_var_ann): External variable annotations are
> +	unshared too.
> +
> +2006-12-22  Kazu Hirata  <kazu@codesourcery.com>
> +
> +	* basic-block.h: Remove the prototype for
> +	commit_edge_insertions_watch_calls.
> + 	* cfgrtl.c (commit_edge_insertion): Drop the last argument.
> +	Simplify.
> +	(commit_edge_insertions_watch_calls): Remove.
> +	(commit_edge_insertions): Adjust the call to
> +	commit_one_edge_insertion.
> +
> +2006-12-22  Zdenek Dvorak <dvorakz@suse.cz>
> +
> +	* tree-ssa-loop-niter.c (zero_p, nonzero_p): Removed.
> +	(number_of_iterations_ne, number_of_iterations_lt_to_ne,
> +	assert_no_overflow_lt, assert_loop_rolls_lt,
> +	number_of_iterations_lt, number_of_iterations_le,
> +	number_of_iterations_cond, tree_simplify_using_condition_1,
> +	number_of_iterations_exit, find_loop_niter, loop_niter_by_eval,
> +	implies_nonnegative_p, implies_ge_p, record_nonwrapping_iv,
> +	idx_infer_loop_bounds, n_of_executions_at_most, scev_probably_wraps_p):
> +	Do not use zero_p/nonzero_p.
> +	* tree-ssa-loop-manip.c (determine_exit_conditions): Ditto.
> +	* tree-ssa-loop-ivopts.c (niter_for_exit, determine_biv_step,
> +	find_interesting_uses_op, find_interesting_uses_cond,
> +	find_interesting_uses_address, find_interesting_uses_stmt,
> +	strip_offset_1, add_candidate_1, add_old_ivs_candidates,
> +	difference_cost, determine_use_iv_cost_condition,
> +	rewrite_use_compare, remove_unused_ivs): Ditto.
> +	* tree-ssa-address.c (tree_mem_ref_addr, create_mem_ref_raw): Ditto.
> +	* tree-ssa-loop-prefetch.c (idx_analyze_ref): Ditto.
> +	* tree-cfg.c (find_taken_edge_cond_expr): Ditto.
> +	* tree.h (zero_p): Declaration removed.
> +	(null_or_integer_zerop, nonnull_and_integer_nonzerop): New.
> +
> +2006-12-22  Manuel Lopez-Ibanez  <manu@gcc.gnu.org>
> +
> +	PR middle-end/7651
> +	* c.opt (Wclobbered): New.
> +	* doc/invoke.texi (Wclobbered): Document it.
> +	(Wextra): Enabled by -Wextra.
> +	* c-opts.c (c_common_post_options): Enabled by -Wextra.
> +	* flow.c (rest_of_handle_life): Replace Wextra with Wclobbered.
> +	* function.c (setjmp_vars_warning): Only warn for Wclobbered.
> +	(setjmp_args_warning): Likewise.
> +	
> +2006-12-22  Kazu Hirata  <kazu@codesourcery.com>
> +
> +	* config/elfos.h, config/spu/spu.c, tree-ssa-operands.h,
> +	tree-ssa-ter.c: Fix comment typos.
> +
> +2006-12-22  Paolo Bonzini  <bonzini@gnu.org>
> +
> +	PR rtl-optimization/29840
> +
> +	* fwprop.c (forward_propagate_into): Reject artificial uses/defs.
> +	(fwprop_init): Add DF_HARD_REGS to df_init call.
> +
>  2006-12-21  Andrew Pinski  <pinskia@gmail.com>
>  
>  	* tree-nested.c (create_tmp_var_for): Check for vector type
> 
> 
> The results can be reproduced by building a compiler with
> 
> --enable-gather-detailed-mem-stats targetting x86-64
> 
> and compiling preprocessed combine.c or testcase from PR8632 with:
> 
> -fmem-report --param=ggc-min-heapsize=1024 --param=ggc-min-expand=1 -Ox -Q
> 
> The memory consumption summary appears in the dump after detailed listing
> of the places they are allocated in.  Peak memory consumption is actually
> computed by looking for maximal value in {GC XXXX -> YYYY} report.
> 
> Your testing script.