Some aspect of GCC memory consumption increased by recent patch

[gcctest at suse dot de]

Hi,
Comparing memory consumption on compilation of combine.i and generate-3.4.ii I got:


comparing combine.c compilation at -O0 level:
Peak amount of GGC memory allocated before garbage collecting run decreased from 12120k to 9500k, overall -27.58%
Peak amount of GGC memory still allocated after garbage collecting decreased from 11641k to 8812k, overall -32.10%
Amount of produced GGC garbage increased from 42976k to 46350k, overall 7.85%
Amount of memory still referenced at the end of compilation decreased from 10910k to 7371k, overall -48.01%
Peak memory use before GGC: 12120k -> 9500k
Peak memory use after GGC: 11641k -> 8812k
Maximum of released memory in single GGC run: 2368k -> 2881k
Garbage: 42976k -> 46350k
Leak: 7371k -> 10910k
Overhead: 5810k -> 5811k
GGC runs: 447 -> 463

comparing combine.c compilation at -O1 level:
Peak amount of GGC memory allocated before garbage collecting run decreased from 10927k to 9702k, overall -12.63%
Peak amount of GGC memory still allocated after garbage collecting decreased from 10454k to 8824k, overall -18.47%
Amount of produced GGC garbage increased from 80911k to 83770k, overall 3.53%
Amount of memory still referenced at the end of compilation decreased from 10006k to 7181k, overall -39.33%
Peak memory use before GGC: 10927k -> 9702k
Peak memory use after GGC: 10454k -> 8824k
Maximum of released memory in single GGC run: 1768k -> 2070k
Garbage: 80911k -> 83770k
Leak: 7181k -> 10006k
Overhead: 13988k -> 14003k
GGC runs: 656 -> 691

comparing combine.c compilation at -O2 level:
Amount of produced GGC garbage increased from 96732k to 99727k, overall 3.10%
Amount of memory still referenced at the end of compilation decreased from 10025k to 6999k, overall -43.23%
Peak memory use before GGC: 13349k -> 13294k
Peak memory use after GGC: 13160k -> 13160k
Maximum of released memory in single GGC run: 2104k -> 2576k
Garbage: 96732k -> 99727k
Leak: 6999k -> 10025k
Overhead: 18848k -> 18818k
GGC runs: 660 -> 678

comparing combine.c compilation at -O3 level:
Peak amount of GGC memory allocated before garbage collecting run decreased from 16951k to 13850k, overall -22.39%
Peak amount of GGC memory still allocated after garbage collecting decreased from 15529k to 13161k, overall -17.99%
Amount of produced GGC garbage increased from 126751k to 131515k, overall 3.76%
Amount of memory still referenced at the end of compilation decreased from 12211k to 7549k, overall -61.75%
Peak memory use before GGC: 16951k -> 13850k
Peak memory use after GGC: 15529k -> 13161k
Maximum of released memory in single GGC run: 3047k -> 3484k
Garbage: 126751k -> 131515k
Leak: 7549k -> 12211k
Overhead: 24703k -> 24734k
GGC runs: 719 -> 743

comparing Gerald's testcase PR8361 compilation at -O0 level:
Amount of produced GGC garbage increased from 255121k to 266369k, overall 4.41%
Amount of memory still referenced at the end of compilation decreased from 72853k to 61513k, overall -18.43%
Peak memory use before GGC: 93183k -> 93183k
Peak memory use after GGC: 91694k -> 91694k
Maximum of released memory in single GGC run: 20289k -> 20289k
Garbage: 255121k -> 266369k
Leak: 61513k -> 72853k
Overhead: 34608k -> 34609k
GGC runs: 538 -> 547

comparing Gerald's testcase PR8361 compilation at -O1 level:
Amount of produced GGC garbage decreased from 659747k to 545183k, overall -21.01%
Amount of memory still referenced at the end of compilation decreased from 90181k to 63318k, overall -42.42%
Peak memory use before GGC: 97174k -> 97174k
Peak memory use after GGC: 90561k -> 90561k
Maximum of released memory in single GGC run: 19488k -> 19488k
Garbage: 659747k -> 545183k
Leak: 63318k -> 90181k
Overhead: 144783k -> 75401k
GGC runs: 808 -> 763

comparing Gerald's testcase PR8361 compilation at -O2 level:
Amount of produced GGC garbage decreased from 726054k to 600944k, overall -20.82%
Amount of memory still referenced at the end of compilation decreased from 89222k to 63940k, overall -39.54%
Peak memory use before GGC: 97175k -> 97175k
Peak memory use after GGC: 90562k -> 90562k
Maximum of released memory in single GGC run: 19489k -> 19489k
Garbage: 726054k -> 600944k
Leak: 63940k -> 89222k
Overhead: 167496k -> 93876k
GGC runs: 852 -> 822

comparing Gerald's testcase PR8361 compilation at -O3 level:
Amount of produced GGC garbage decreased from 775639k to 617007k, overall -25.71%
Amount of memory still referenced at the end of compilation decreased from 90657k to 64593k, overall -40.35%
Peak memory use before GGC: 92864k -> 92864k
Peak memory use after GGC: 91396k -> 91396k
Maximum of released memory in single GGC run: 20194k -> 20194k
Garbage: 775639k -> 617007k
Leak: 64593k -> 90657k
Overhead: 182029k -> 95859k
GGC runs: 848 -> 803

Head of changelog is:

--- /usr/src/SpecTests/sandbox-britten-memory/x86_64/mem-result/ChangeLog	2004-09-08 06:01:21.000000000 +0000
+++ /usr/src/SpecTests/sandbox-britten-memory/gcc/gcc/ChangeLog	2004-09-08 11:38:37.000000000 +0000
@@ -1,3 +1,136 @@
+2004-09-08  Jie Zhang  <zhangjie@magima.com.cn>
+
+	* tree-ssa-alias.c (compute_flow_insensitive_aliasing): If type
+	memory tag is call clobbered, so are its aliases.
+	(group_aliases): When two memory tags being grouped, if one is
+	call clobbered, so are the other and its aliases.
+	(add_may_alias): Remove call-clobbering stuff.
+	(replace_may_alias): Likewise.
+	(merge_pointed_to_info): Merge pt_global_mem
+
+2004-09-08  Jan Hubicka  <jh@suse.cz>
+
+	* cgraph.c (cgraph_remove_node): Free DECL_INITIAL field of node.
+	* cgraphunit.c (verify_cgraph): Don't verify on syntax errors.
+	(cgraph_expand_function): Remove stale cgraph edges of currently
+	compiled function; fix non-unit-at-a-time code copying function
+	node for later reuse.
+
+2004-09-08  Nathan Sidwell  <nathan@codesourcery.com>
+
+	* vec.c (vec_p_reserve, vec_o_reserve): Rename to ...
+	(vec_gc_p_reserve, vec_gc_o_reserve): ... here. Clone to
+	(vec_heap_p_reserve, vec_heap_o_reserve): ... here, adjust.
+	(vec_gc_free, vec_heap_free): New.
+	* vec.h (DEF_VEC_GC_P, DEF_VEC_MALLOC_P): New.
+	(DEF_VEC_P): Add allocator argument. Adjust.
+	(DEF_VEC_GC_O, DEF_VEC_MALLOC_O): New.
+	(DEF_VEC_O): Add allocator argument. Adjust.
+	(VEC(free)): New.
+
+	* tree.h (tree): Define a GC'd vector.
+	* lamba-code.c (lambda_loop): Likewise.
+	* value-prof.h (histogram_value): Likewise.
+
+2004-09-08  Nathan Sidwell  <nathan@codesourcery.com>
+
+	* emit-rtl.c (immed_double_const): Use gcc_assert and gcc_unreachable.
+	(gen_rtx_SUBREG, gen_reg_rtx, mark_user_reg, subreg_hard_regno,
+	gen_lowpart_common, gen_highpart, gen_highpart_mode,
+	subreg_highpart_offset, operand_subword, operand_subword_force,
+	mem_expr_equal_p, set_mem_attributes_minus_bitpos,
+	set_mem_alias_set, change_address_1, verify_rtx_sharing,
+	copy_most_rtx, set_first_insn, set_last_insn, prev_cc0_setter,
+	try_split, add_insn_after, add_insn_before, remove_insn,
+	remove_unnecessary_notes, emit_insn_before, emit_jump_insn_before,
+	emit_call_insn_before, emit_insn_after, emit_jump_insn_after,
+	emit_call_insn_after, emit_insn, emit_jump_insn, emit_call_insn,
+	set_unique_reg_note, emit, push_to_full_sequence, copy_insn_1,
+	gen_const_vector_0, emit_copy_of_insn_after): Likewise.
+	* et-forest.c (set_prev, set_next, et_check_occ_sanity,
+	record_path_before_1, check_path_after_1, check_path_after): Likewise.
+	* except.c (gen_eh_region, resolve_one_fixup_region,
+	remove_unreachable_regions, convert_from_eh_region_ranges_1,
+	add_ehl_entry, duplicate_eh_region_1, build_post_landing_pads,
+	connect_post_landing_pads, sjlj_emit_function_exit,
+	remove_exception_handler_label, remove_eh_handler,
+	reachable_next_level, collect_one_action_chain,
+	output_function_exception_table): Likewise.
+	* explow.c (trunc_int_for_mode, copy_to_mode_reg,
+	optimize_save_area_alloca, allocate_dynamic_stack_space,
+	probe_stack_range, hard_function_value): Likewise.
+	* expmed.c (mode_for_extraction, store_bit_field,
+	store_fixed_bit_field, extract_bit_field, expand_shift,
+	expand_mult_const, expand_mult, choose_multiplier,
+	expand_mult_highpart, expand_divmod, emit_store_flag,
+	do_cmp_and_jump): Likewise.
+	* expr.c (convert_move, convert_modes, move_by_pieces,
+	move_by_pieces_ninsns, move_by_pieces_1, emit_block_move,
+	move_block_from_reg, gen_group_rtx, emit_group_load,
+	emit_group_move, emit_group_store, use_reg, use_regs,
+	can_store_by_pieces, store_by_pieces, store_by_pieces_1,
+	emit_move_insn, emit_move_insn_1, emit_push_insn,
+	expand_assignment, store_expr, count_type_elements,
+	store_constructor, store_field, safe_from_p, expand_var,
+	expand_expr_addr_expr, expand_expr_real_1, do_store_flag): Likewise.
+
+2004-09-08  Nathan Sidwell  <nathan@codesourcery.com>
+
+	* dbxout.c (dbxout_type, dbxout_type_name, dbxout_symbol): Use
+	gcc_assert and gcc_unreachable.
+	* ddg.c (create_ddg_dependence, add_deps_for_def,
+	add_deps_for_use, create_ddg, add_edge_to_ddg): Likewise.
+	* df.c (df_ref_unlink, df_ref_record, df_uses_record,
+	df_reg_def_chain_create, df_reg_use_chain_create, df_analyze,
+	df_insn_delete, df_refs_reg_replace, df_ref_reg_replace,
+	df_insns_modify, df_pattern_emit_before, df_bb_reg_live_start_p,
+	df_bb_reg_live_end_p, df_bb_regs_lives_compare,
+	df_bb_single_def_use_insn_find, dataflow_set_a_op_b,
+	dataflow_set_copy, hybrid_search, diagnostic.c,
+	diagnostic_build_prefix, diagnostic_count_diagnostic): Likewise.
+	* dojump.c (do_jump): Likewise.
+	* dominance.c (calc_dfs_tree_nonrec, calc_dfs_tree,
+	compute_dom_fast_query, calculate_dominance_info,
+	free_dominance_info, get_immediate_dominator,
+	set_immediate_dominator, get_dominated_by,
+	redirect_immediate_dominators, nearest_common_dominator,
+	dominated_by_p, verify_dominators, recount_dominator,
+	iterate_fix_dominators, add_to_dominance_info,
+	delete_from_dominance_info): Likewise.
+	* dwarf2asm.c (size_of_encoded_value, eh_data_format_name,
+	dw2_asm_output_delta_uleb128, dw2_asm_output_delta_sleb128,
+	dw2_force_const_mem, dw2_asm_output_encoded_addr_rtx): Likewise.
+	* dwarf2out.c (expand_builtin_init_dwarf_reg_sizes, reg_save,
+	initial_return_save, stack_adjust_offset, dwarf2out_stack_adjust,
+	flush_queued_reg_saves, dwarf2out_frame_debug_expr,
+	dwarf2out_frame_debug, dw_cfi_oprnd1_desc, output_cfi,
+	output_call_frame_info, output_loc_operands, build_cfa_loc,
+	decl_ultimate_origin, AT_flag, AT_int, AT_unsigned, AT_string,
+	AT_string_form, add_AT_specification, AT_ref, set_AT_ref_external,
+	AT_loc, AT_loc_list, AT_addr, AT_lbl, add_child_die,
+	splice_child_die, attr_checksum, same_dw_val_p,
+	break_out_includes, build_abbrev_table, size_of_die, mark_dies,
+	unmark_dies, value_format, output_loc_list, output_die,
+	output_pubnames, output_aranges, base_type_die, is_base_type,
+	modified_type_die, dbx_reg_number, multiple_reg_loc_descriptor,
+	mem_loc_descriptor, loc_descriptor, loc_descriptor_from_tree_1,
+	field_byte_offset, add_data_member_location_attribute,
+	add_const_value_attribute, rtl_for_decl_location,
+	add_location_or_const_value_attribute, add_byte_size_attribute,
+	add_bit_offset_attribute, add_bit_size_attribute,
+	add_abstract_origin_attribute, pop_decl_scope, scope_die_for,
+	decl_start_label, gen_formal_parameter_die,
+	gen_type_die_for_member, gen_subprogram_die, gen_label_die,
+	gen_typedef_die, gen_type_die, gen_tagged_type_instantiation_die,
+	force_decl_die, force_type_die, gen_decl_die,
+	dwarf2out_imported_module_or_decl, prune_unused_types_prune,
+	dwarf2out_finish): Likewise.
+
+2004-09-08  Andreas Tobler  <toa@pop.agri.ch>
+
+	* Makefile.in (builtins.o): Depend on tree-mudflap.h.
+	* builtins.c: Include tree-mudflap.h.
+
 2004-09-08  Uros Bizjak  <uros@kss-loka.si>
 
 	* config/i386/i386.c (ix86_prepare_fp_compare_args): Do not
--- /usr/src/SpecTests/sandbox-britten-memory/x86_64/mem-result/ChangeLog.cp	2004-09-06 23:57:27.000000000 +0000
+++ /usr/src/SpecTests/sandbox-britten-memory/gcc/gcc/cp/ChangeLog	2004-09-08 11:38:44.000000000 +0000
@@ -1,3 +1,9 @@
+2004-09-08  Nathan Sidwell  <nathan@codesourcery.com>
+
+	* cp-tree.h (tree_pair_s): Define a GC'd vector.
+	* name-lookup.h (cxx_saved_binding, cp_class_binding): Likewise.
+	* semantics.c (deferred_access): Likewise.
+
 2004-09-06  Daniel Jacobowitz  <dan@debian.org>
 
 	* semantics.c (expand_body): Assert that we are not nested.

I am friendly script caring about memory consumption in GCC.  Please contact
jh@suse.cz if something is going wrong.

The results can be reproduced by building compiler with
--enable-gather-detailed-mem-stats targetting x86-64 and compiling preprocessed
combine.c or testcase from PR8632 with:

-fmem-report --param=ggc-min-heapsize=1024 --param=ggc-min-expand=1 -Ox -Q

The memory consumption summary appears in the dump after detailed listing of
the places they are allocated in.  Peak memory consumption is actually computed
by looking for maximal value in {GC XXXX -> YYYY} report.

Yours testing script.