User account creation filtered due to spam.
Here is a code which doesn't compile. It takes all swap space (around 2GB) and all system memory(about 1GB), compiles around 15-20 minutes and then crashes with the following message : "g++: Internal error: Killed (program cc1plus)"
Created attachment 12409 [details] preprocessed source file This is output of the compiler with the -save-temps switch. It's compressed 'cause there is a 1 MB limit on the attachement size.
Created attachment 12410 [details] gcc -V output version and configuration information on the used gcc.
Created attachment 12411 [details] Command Line and Error Message
This takes 79% of my 2GB of memory.
Confirmed. Uses a lot of memory.
Execution times (seconds) garbage collection : 0.88 ( 1%) usr 0.00 ( 0%) sys 0.87 ( 1%) wall 0 kB ( 0%) ggc preprocessing : 0.10 ( 0%) usr 0.04 ( 1%) sys 0.22 ( 0%) wall 796 kB ( 0%) ggc parser : 17.90 (14%) usr 0.77 (15%) sys 19.39 (15%) wall 325217 kB (64%) ggc name lookup : 94.51 (76%) usr 4.25 (84%) sys 98.43 (75%) wall 175474 kB (34%) ggc tree gimplify : 0.07 ( 0%) usr 0.00 ( 0%) sys 0.04 ( 0%) wall 2548 kB ( 1%) ggc varconst : 11.33 ( 9%) usr 0.01 ( 0%) sys 11.33 ( 9%) wall 2833 kB ( 1%) ggc symout : 0.00 ( 0%) usr 0.00 ( 0%) sys 0.01 ( 0%) wall 32 kB ( 0%) ggc TOTAL : 124.80 5.08 130.45 508745 kB and using peak 1.7GB ram (x86_64 with -m32) on the trunk. I guess we're just not collecting garbage during parsing at all. Note that I get errors from compiling the testcase though: [... very large template ...] test_basic_metafunctions.hpp:176: instantiated from here ../src/sd/../static_component.hpp:91: error: 'process_outputs' is not a member of 'mpl_::void_' (4.1 also isn't happy with the testcase) Can you by any chance provide a testcase that compiles ok with at least gcc 4.1.x? (even better mainline...) Thanks! Profiling the bugger yields: Flat profile: Each sample counts as 0.01 seconds. % cumulative self self total time seconds seconds calls Ks/call Ks/call name 12.44 178.30 178.30 121375686 0.00 0.00 comptypes 9.29 311.48 133.18 218375227 0.00 0.00 dump_aggr_type 8.69 436.02 124.54 155559998 0.00 0.00 comp_template_args 8.59 559.19 123.17 620924334 0.00 0.00 pp_base_append_text 7.85 671.72 112.53 1278486 0.00 0.00 retrieve_specialization 6.92 770.99 99.27 173585104 0.00 0.00 template_args_equal 6.54 864.80 93.81 462049326 0.00 0.00 pp_base_character 5.05 937.24 72.44 67358 0.00 0.00 ht_lookup 3.32 984.79 47.55 620924334 0.00 0.00 pp_base_string 3.13 1029.71 44.92 186232097 0.00 0.00 dump_decl 2.58 1066.65 36.95 218532269 0.00 0.00 pp_c_type_qualifier_list 2.36 1100.46 33.81 16289532 0.00 0.00 dump_template_parms 2.32 1133.71 33.25 404449103 0.00 0.00 dump_scope 2.28 1166.40 32.69 218677178 0.00 0.00 dump_type 1.95 1194.40 28.00 404642903 0.00 0.00 pp_c_identifier 1.78 1219.92 25.53 218375227 0.00 0.00 class_key_or_enum_as_string 1.38 1239.65 19.73 216176146 0.00 0.00 dump_template_argument 1.03 1254.37 14.72 3919144 0.00 0.00 purpose_member 0.88 1266.94 12.57 108004962 0.00 0.00 pp_base_last_position_in_text 0.68 1276.74 9.80 162182109 0.00 0.00 pp_cxx_separate_with 0.61 1285.53 8.80 202214181 0.00 0.00 pp_cxx_colon_colon 0.41 1291.38 5.85 54002002 0.00 0.00 pp_cxx_end_template_argument_list 0.41 1297.19 5.81 5800051 0.00 0.00 ggc_alloc_stat 0.40 1302.92 5.73 4113746 0.00 0.00 tsubst 0.37 1308.20 5.29 54002002 0.00 0.00 pp_cxx_begin_template_argument_list 0.31 1312.58 4.38 301296 0.00 0.00 pp_base_emit_prefix 0.30 1316.88 4.30 450268 0.00 0.00 gt_ggc_mx_lang_tree_node 0.27 1320.81 3.93 302601 0.00 0.00 reinit_cxx_pp 0.27 1324.73 3.92 6289194 0.00 0.00 ggc_set_mark 0.26 1328.46 3.73 21 0.00 0.00 pp_base_newline 0.25 1332.04 3.59 892391 0.00 0.00 lookup_template_class 0.24 1335.43 3.39 14090587 0.00 0.00 pp_c_constant 0.20 1338.34 2.91 67465 0.00 0.00 push_to_top_level 0.20 1341.19 2.85 2159736 0.00 0.00 dfs_walk_all 0.17 1343.61 2.43 935808 0.00 0.00 coerce_template_parms 0.17 1346.03 2.42 1554163 0.00 0.00 lookup_field_1 0.16 1348.27 2.24 10344444 0.00 0.00 cp_tree_equal 0.15 1350.49 2.22 222604 0.00 0.00 ht_lookup_with_hash
Created attachment 12415 [details] preprocessed source file (version 2) The previous test case had common C++ error in it, which prevents it from being compiled properly. Try this one. It generates the same "internal error" message, but I think that it's free from the C++ errors. If not, I'll make some steps back and provide an example which also requires a lot of memory to compile, but compiles properly, without "internal error".
No success yet: test_basic_metafunctions.hpp:176: instantiated from here ../src/sd/../static_component.hpp:57: error: 'process_outputs' is not a member of 'mpl_::void_' and other errors.
Created attachment 12420 [details] preprocessed source file (version 3) Compiles without errors on my laptop, but still eats a LOT of memory.
Subject: Re: using boost::MPL requires lots of memory Try preprocessed source file v3. It compiles without error on my laptop, but still takes a LOT of memory and time to compile. On 12 Oct 2006 13:09:02 -0000, rguenth at gcc dot gnu dot org <gcc-bugzilla@gcc.gnu.org> wrote: > > > ------- Comment #8 from rguenth at gcc dot gnu dot org 2006-10-12 13:09 ------- > No success yet: > > test_basic_metafunctions.hpp:176: instantiated from here > ../src/sd/../static_component.hpp:57: error: 'process_outputs' is not a member > of 'mpl_::void_' > > and other errors. > > > -- > > > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=29433 > > ------- You are receiving this mail because: ------- > You reported the bug, or are watching the reporter. >
That works. Updated profile: Flat profile: Each sample counts as 0.01 seconds. % cumulative self self total time seconds seconds calls Ks/call Ks/call name 11.26 131.84 131.84 77379135 0.00 0.00 comptypes 11.05 261.22 129.38 153228836 0.00 0.00 dump_aggr_type 7.47 348.70 87.48 441675354 0.00 0.00 pp_base_append_text 7.06 431.38 82.68 101695144 0.00 0.00 comp_template_args 6.90 512.09 80.71 870176 0.00 0.00 retrieve_specialization 6.09 583.32 71.24 116984878 0.00 0.00 template_args_equal 4.82 639.78 56.46 325283356 0.00 0.00 pp_base_character 4.56 693.10 53.32 59058 0.00 0.00 ht_lookup 3.63 735.58 42.48 441675354 0.00 0.00 pp_base_string 3.30 774.18 38.60 287071962 0.00 0.00 dump_scope 3.20 811.63 37.46 10634496 0.00 0.00 dump_template_parms 2.99 846.65 35.02 133934524 0.00 0.00 dump_decl 2.82 879.71 33.06 153423130 0.00 0.00 dump_type 2.25 905.99 26.29 153344660 0.00 0.00 pp_c_type_qualifier_list 1.97 929.03 23.04 152685422 0.00 0.00 dump_template_argument 1.96 951.95 22.92 287215230 0.00 0.00 pp_c_identifier 1.35 967.81 15.86 3824263 0.00 0.00 purpose_member 1.34 983.49 15.68 153228836 0.00 0.00 class_key_or_enum_as_string 0.77 992.52 9.03 115629446 0.00 0.00 pp_cxx_separate_with 0.68 1000.52 8.00 144377450 0.00 0.00 pp_cxx_colon_colon 0.60 1007.53 7.01 74128630 0.00 0.00 pp_base_last_position_in_text 0.47 1013.00 5.47 4619635 0.00 0.00 ggc_alloc_stat 0.43 1017.98 4.98 37063838 0.00 0.00 pp_cxx_end_template_argument_list 0.42 1022.89 4.92 2734603 0.00 0.00 tsubst 0.35 1026.98 4.09 37063838 0.00 0.00 pp_cxx_begin_template_argument_list 0.35 1031.07 4.09 574289 0.00 0.00 gt_ggc_mx_lang_tree_node 0.33 1034.88 3.81 218258 0.00 0.00 pp_base_emit_prefix 0.31 1038.56 3.68 6251202 0.00 0.00 ggc_set_mark 0.31 1042.21 3.66 10098142 0.00 0.00 pp_c_constant 0.28 1045.54 3.33 1473688 0.00 0.00 dfs_walk_all 0.28 1048.79 3.26 588277 0.00 0.00 lookup_template_class 0.27 1051.95 3.16 pp_base_newline 0.23 1054.67 2.72 10081686 0.00 0.00 pp_c_integer_constant 0.22 1057.22 2.55 1172353 0.00 0.00 lookup_field_1 0.22 1059.75 2.53 10101968 0.00 0.00 dump_expr 0.21 1062.20 2.45 4021108 0.00 0.00 dfs_access_in_type 0.21 1064.64 2.44 48588 0.00 0.00 push_to_top_level 0.20 1066.97 2.34 8942686 0.00 0.00 cp_tree_equal 0.18 1069.12 2.15 615699 0.00 0.00 coerce_template_parms 0.17 1071.14 2.03 219438 0.00 0.00 reinit_cxx_pp 0.17 1073.11 1.97 214436 0.00 0.00 ht_lookup_with_hash 0.16 1075.00 1.89 128624 0.00 0.00 get_expr_operands 0.14 1076.60 1.60 609605 0.00 0.00 tsubst_aggr_type 0.13 1078.18 1.58 2662882 0.00 0.00 store_binding 0.13 1079.75 1.57 587010 0.00 0.00 instantiate_class_template 0.12 1081.19 1.45 2 0.00 0.00 dump_function_decl 0.12 1082.63 1.44 928019 0.00 0.00 tree_cons_stat 0.12 1084.03 1.40 709984 0.00 0.00 tsubst_template_args 0.12 1085.40 1.38 4639830 0.00 0.00 context_for_name_lookup 0.10 1086.59 1.19 2314384 0.00 0.00 htab_find_with_hash 0.09 1087.68 1.09 475135 0.00 0.00 lookup_fnfields_1 0.09 1088.74 1.06 42858 0.00 0.00 finish_struct_1 0.09 1089.78 1.04 695522 0.00 0.00 htab_find_slot_with_hash 0.09 1090.81 1.03 10098142 0.00 0.00 pp_cxx_constant 0.09 1091.83 1.02 pp_c_conditional_expression
-O2 -ftime-report: Execution times (seconds) parser : 11.82 (14%) usr 0.54 (15%) sys 12.81 (14%) wall 254865 kB (62%) ggc name lookup : 62.00 (74%) usr 3.04 (82%) sys 64.50 (73%) wall 123819 kB (30%) ggc varconst : 7.37 ( 9%) usr 0.00 ( 0%) sys 7.50 ( 8%) wall 2369 kB ( 1%) ggc TOTAL : 84.28 3.71 88.39 411895 kB ! (profile was also -O2) Not too much code is generated by the testcase (-O2): > size main.o text data bss dec hex filename 21680 4 76 21760 5500 main.o
Subject: Re: using boost::MPL requires lots of memory But my main target is the previous testcase. It should at least provide me a C++ error, but not crash with internal error. On 13 Oct 2006 11:05:35 -0000, rguenth at gcc dot gnu dot org <gcc-bugzilla@gcc.gnu.org> wrote: > > > ------- Comment #12 from rguenth at gcc dot gnu dot org 2006-10-13 11:05 ------- > -O2 -ftime-report: > > Execution times (seconds) > parser : 11.82 (14%) usr 0.54 (15%) sys 12.81 (14%) wall > 254865 kB (62%) ggc > name lookup : 62.00 (74%) usr 3.04 (82%) sys 64.50 (73%) wall > 123819 kB (30%) ggc > varconst : 7.37 ( 9%) usr 0.00 ( 0%) sys 7.50 ( 8%) wall > 2369 kB ( 1%) ggc > TOTAL : 84.28 3.71 88.39 > 411895 kB > > ! (profile was also -O2) > > Not too much code is generated by the testcase (-O2): > > > size main.o > text data bss dec hex filename > 21680 4 76 21760 5500 main.o > > > -- > > > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=29433 > > ------- You are receiving this mail because: ------- > You reported the bug, or are watching the reporter. >
Sure, but the crash is due to gcc running out of memory. And the third testcase is a good one to attack memory usage and compile-time problems on all of the three testcases. But don't hold your breath...
Subject: Re: using boost::MPL requires lots of memory Ok, if there is anything else I could help, please mail me. On 13 Oct 2006 12:18:51 -0000, rguenth at gcc dot gnu dot org <gcc-bugzilla@gcc.gnu.org> wrote: > > > ------- Comment #14 from rguenth at gcc dot gnu dot org 2006-10-13 12:18 ------- > Sure, but the crash is due to gcc running out of memory. And the third > testcase is a good one to attack memory usage and compile-time problems on all > of the three testcases. > > But don't hold your breath... > > > -- > > > http://gcc.gnu.org/bugzilla/show_bug.cgi?id=29433 > > ------- You are receiving this mail because: ------- > You reported the bug, or are watching the reporter. >
The bootleneck is clearly using the diagnostic machinery for setting up DECL_NAME of the type decls. I wonder if we cannot directly compute and store mangled names in DECL_NAME which would save both time and memory... Any ideas? Mark? Jason?
Created attachment 12666 [details] testcase run through uninclude unincluded so it's 64bit clean also.
Most of the memory used is spent in the string pool: String pool entries 68980 identifiers 68980 (100.00%) slots 131072 bytes 1012M (10M overhead) table size 1024k coll/search 0.7816 ins/search 0.2876 avg. entry 15394.37 bytes (+/- 68924.74) longest entry 2130575 if you add -g to the compilation, dwarf2out.c happily xstrdup()s most of them a few times and memory consumption goes up to 3.9GB. All the stringpool handling and it's usage by classtype_mangled_name is not very well optimized for such large strings (after all, we know the string length there for example).
Subject: Bug number PR29433 A patch for this bug has been added to the patch tracker. The mailing list url for the patch is http://gcc.gnu.org/ml/gcc-patches/2006-11/msg01900.html
Subject: Bug number PR29433 A patch for this bug has been added to the patch tracker. The mailing list url for the patch is http://gcc.gnu.org/ml/gcc-patches/2006-11/msg02053.html
Subject: Bug 29433 Author: rguenth Date: Fri Dec 1 16:37:38 2006 New Revision: 119404 URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=119404 Log: 2006-12-01 Richard Guenther <rguenther@suse.de> PR c++/29433 * dwarf2out.c (struct pubname_struct): Make name const. (add_pubtype): Do not xstrdup identifiers. Modified: trunk/gcc/ChangeLog trunk/gcc/dwarf2out.c
Improved for the case of building with -g, which should now be as bad as with plain -O0.
Fixed.
*** Bug 32290 has been marked as a duplicate of this bug. ***