Bug 77844 - [5 Regression] Compilation of simple C++ example exhaust memory
Summary: [5 Regression] Compilation of simple C++ example exhaust memory
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: debug (show other bugs)
Version: 4.9.4
: P2 normal
Target Milestone: 7.0
Assignee: Jakub Jelinek
URL:
Keywords: compile-time-hog, memory-hog
Depends on:
Blocks:
 
Reported: 2016-10-04 15:02 UTC by graeser
Modified: 2018-10-13 18:04 UTC (History)
2 users (show)

See Also:
Host:
Target:
Build:
Known to work: 6.1.0
Known to fail: 5.4.0, 7.0
Last reconfirmed: 2016-10-05 00:00:00

Attachments
Compiling with 'g++-4.9 -O -g' triggers the problem. (212 bytes, text/x-csrc)
2016-10-04 15:02 UTC, graeser 		Details
gcc7-pr77844.patch (926 bytes, patch)
2016-12-12 16:33 UTC, Jakub Jelinek 		Details | Diff
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description graeser 2016-10-04 15:02:07 UTC 
Created attachment 39750 [details]
Compiling with 'g++-4.9 -O -g' triggers the problem.

When trying to compile the attached example with 'g++-4.9 -O -g' compilation does not terminate and memory usage grows slowly. I terminated compilation when gcc used more than 14gb of memory.

Notice that the issue is not triggered if one level of indirection is removed, e.g., if the body of test() is copied into main().
Comment 1 Andrew Pinski 2016-10-04 15:11:09 UTC 
This might have been fixed already. Can you try out gcc 5.4.0?
Comment 2 graeser 2016-10-04 15:31:51 UTC 
With gcc 5.4.0 on Ubuntu this issue seems to be essentially fixed. However, compilation still takes about 14s and eats up about 1gb which looks huge compared to 0.1s and 31mb if the indirection via test() is removed.
Comment 3 Martin Liška 2016-10-05 08:32:46 UTC 
Can't reproduce the problem, there are times I get on my i7-4770:

gcc_bisect.py bisect 'g++ pr77844.cpp -O -g'
Releases
  4.5.0 (72d56ebb66ff343f779686a70c7be429238f5d5d)(14 Apr 2010 09:22): [took: 0.100s] running command with result: OK
  4.5.1 (1f8be1d60ef7e4aba6fc9c289312a23478660404)(31 Jul 2010 09:26): [took: 0.098s] running command with result: OK
  4.5.2 (3598ee5d488243596fba837438824f6e7ab0b035)(16 Dec 2010 12:33): [took: 0.099s] running command with result: OK
  4.5.3 (db65a9d18789609ec70d37f0bb956dd4d6420417)(28 Apr 2011 14:12): [took: 0.097s] running command with result: OK
  4.5.4 (0c2ca17227668950cae3876c44038c69bade181c)(02 Jul 2012 09:28): [took: 0.097s] running command with result: OK
  4.6.0 (cc52afb59a7d680efc31b952a328b1d53022d8bc)(25 Mar 2011 16:57): [took: 0.097s] running command with result: OK
  4.6.1 (31b6f3a04d5ac5d9b9dddafc62cc4e41beb5e590)(27 Jun 2011 10:05): [took: 0.099s] running command with result: OK
  4.6.2 (e07bbb139795b3a6702c7c00ab80524be0ba92e8)(26 Oct 2011 09:08): [took: 0.096s] running command with result: OK
  4.6.3 (31839765ee7d53766a69f418e9d7a4ed750de4e9)(01 Mar 2012 11:56): [took: 0.098s] running command with result: OK
  4.6.4 (632cb4d5f22f598d1395fbae90a10fb679e06054)(12 Apr 2013 09:52): [took: 0.099s] running command with result: OK
  4.7.0 (93c5ebd73a4d1626d25203081d079cdd68222fcc)(22 Mar 2012 07:11): [took: 0.100s] running command with result: OK
  4.7.1 (0e3097e7d505b7be8b05725988e27b2bea72fa39)(14 Jun 2012 08:32): [took: 0.101s] running command with result: OK
  4.7.2 (c9b304ada7111264d743b1c588393b66a3e3edfb)(20 Sep 2012 06:54): [took: 0.100s] running command with result: OK
  4.7.3 (f22940cb824859bd5dcc1c3cfcf7b535a5ba7084)(11 Apr 2013 07:57): [took: 0.100s] running command with result: OK
  4.7.4 (ae10eb82fe34c18640ad65c2ab94ffc53f349315)(12 Jun 2014 12:08): [took: 0.102s] running command with result: OK
  4.8.0 (e9c762ec4671d77e301492e4f9e92e1d3d667188)(22 Mar 2013 10:05): [took: 0.105s] running command with result: OK
  4.8.1 (caa62b4636bfed7193ef77027f56d34a62047b19)(31 May 2013 09:02): [took: 0.103s] running command with result: OK
  4.8.2 (9bcca88e24e64d4e23636aafa3404088b13bcb0e)(16 Oct 2013 07:20): [took: 0.104s] running command with result: OK
  4.8.3 (6bbf0dec66c0e719b06cd2fe67559fda6df09000)(22 May 2014 09:10): [took: 0.104s] running command with result: OK
  4.8.4 (1a97fa0bb3fa5669f763ab43d6f2b77378b68ff6)(19 Dec 2014 11:43): [took: 0.104s] running command with result: OK
  4.8.5 (cf82a597b0d189857acb34a08725762c4f5afb50)(23 Jun 2015 07:54): [took: 0.106s] running command with result: OK
  4.9.0 (a7aa383874520cd5762701f1c790c930c5ab5bb5)(22 Apr 2014 09:43): [took: 0.112s] running command with result: OK
  4.9.1 (c6fa1b412663593960e6240eb66d82fa41a1fa0b)(16 Jul 2014 10:04): [took: 0.110s] running command with result: OK
  4.9.2 (c1283af40b65f1ad862cf5b27e2d9ed10b2076b6)(30 Oct 2014 08:27): [took: 0.111s] running command with result: OK
  4.9.3 (876d41ed80ce13e060084ed5a552c37c301e5563)(26 Jun 2015 17:57): [took: 0.112s] running command with result: OK
  4.9.4 (d3191480f376c78031aa1ca0f6a54f4cf67f4268)(03 Aug 2016 05:07): [took: 0.113s] running command with result: OK
  5.1.0 (d5ad84b309d0d97d3955fb1f62a96fc262df2b76)(22 Apr 2015 08:43): [took: 0.130s] running command with result: OK
  5.2.0 (7b26e3896e268cd452d68ff8df04f4563d5db6ef)(16 Jul 2015 09:13): [took: 0.130s] running command with result: OK
  5.3.0 (2bc376d60753a58b10cb179f8edb7d72bee7a88b)(04 Dec 2015 10:45): [took: 0.135s] running command with result: OK
  5.4.0 (9d0507742960aa9f2b99bc6e9584655ecc611792)(03 Jun 2016 08:41): [took: 0.137s] running command with result: OK
  6.1.0 (c441d9e8e0438dcf54274ec7a3539859e94ad201)(27 Apr 2016 08:20): [took: 0.181s] running command with result: OK
  6.2.0 (6ac74a62ba7258299cf85fbef9bf45333ddf10c0)(22 Aug 2016 08:01): [took: 0.182s] running command with result: OK
Comment 4 graeser 2016-10-05 09:21:42 UTC 
I'm sorry a typo slipped into the report. You need '-O3 -g' to trigger the error. '-O3' or '-g' alone both work nicely.
Comment 5 Martin Liška 2016-10-05 10:25:14 UTC 
Ok, now I can confirm the hog:

Releases
  4.5.0 (72d56ebb66ff343f)(14 Apr 2010 09:22): [took: 0.190s] result: OK
  4.5.1 (1f8be1d60ef7e4ab)(31 Jul 2010 09:26): [took: 0.195s] result: OK
  4.5.2 (3598ee5d48824359)(16 Dec 2010 12:33): [took: 0.196s] result: OK
  4.5.3 (db65a9d18789609e)(28 Apr 2011 14:12): [took: 0.193s] result: OK
  4.5.4 (0c2ca17227668950)(02 Jul 2012 09:28): [took: 0.196s] result: OK
  4.6.0 (cc52afb59a7d680e)(25 Mar 2011 16:57): [took: 0.160s] result: OK
  4.6.1 (31b6f3a04d5ac5d9)(27 Jun 2011 10:05): [took: 0.161s] result: OK
  4.6.2 (e07bbb139795b3a6)(26 Oct 2011 09:08): [took: 0.164s] result: OK
  4.6.3 (31839765ee7d5376)(01 Mar 2012 11:56): [took: 0.160s] result: OK
  4.6.4 (632cb4d5f22f598d)(12 Apr 2013 09:52): [took: 0.158s] result: OK
  4.7.0 (93c5ebd73a4d1626)(22 Mar 2012 07:11): [took: 0.185s] result: OK
  4.7.1 (0e3097e7d505b7be)(14 Jun 2012 08:32): [took: 0.187s] result: OK
  4.7.2 (c9b304ada7111264)(20 Sep 2012 06:54): [took: 0.190s] result: OK
  4.7.3 (f22940cb824859bd)(11 Apr 2013 07:57): [took: 5.465s] result: OK
  4.7.4 (ae10eb82fe34c186)(12 Jun 2014 12:08): [took: 5.419s] result: OK
  4.8.0 (e9c762ec4671d77e)(22 Mar 2013 10:05): [took: 10.004s] result: FAILED
  4.8.1 (caa62b4636bfed71)(31 May 2013 09:02): [took: 10.004s] result: FAILED
  4.8.2 (9bcca88e24e64d4e)(16 Oct 2013 07:20): [took: 10.004s] result: FAILED
  4.8.3 (6bbf0dec66c0e719)(22 May 2014 09:10): [took: 10.004s] result: FAILED
  4.8.4 (1a97fa0bb3fa5669)(19 Dec 2014 11:43): [took: 10.003s] result: FAILED
  4.8.5 (cf82a597b0d18985)(23 Jun 2015 07:54): [took: 10.003s] result: FAILED
  4.9.0 (a7aa383874520cd5)(22 Apr 2014 09:43): [took: 10.004s] result: FAILED
  4.9.1 (c6fa1b4126635939)(16 Jul 2014 10:04): [took: 10.004s] result: FAILED
  4.9.2 (c1283af40b65f1ad)(30 Oct 2014 08:27): [took: 10.004s] result: FAILED
  4.9.3 (876d41ed80ce13e0)(26 Jun 2015 17:57): [took: 10.004s] result: FAILED
  4.9.4 (d3191480f376c780)(03 Aug 2016 05:07): [took: 10.004s] result: FAILED
  5.1.0 (d5ad84b309d0d97d)(22 Apr 2015 08:43): [took: 10.004s] result: FAILED
  5.2.0 (7b26e3896e268cd4)(16 Jul 2015 09:13): [took: 10.004s] result: FAILED
  5.3.0 (2bc376d60753a58b)(04 Dec 2015 10:45): [took: 10.004s] result: FAILED
  5.4.0 (9d0507742960aa9f)(03 Jun 2016 08:41): [took: 10.004s] result: FAILED
  6.1.0 (c441d9e8e0438dcf)(27 Apr 2016 08:20): [took: 0.293s] result: OK
  6.2.0 (6ac74a62ba725829)(22 Aug 2016 08:01): [took: 0.317s] result: OK

Current trunk is also affected, and the regression started with r236440.
Comment 6 Martin Liška 2016-10-05 10:27:02 UTC 
Tested command: timeout 10 g++ pr77844.cpp -O3 -g
Comment 7 Richard Biener 2016-10-05 12:11:55 UTC 
It's fast with -fno-var-tracking.  We have lots of cse-lim stuff in test() which gets down to a single BB with lots of DEBUG stmts (mostly resets).  I suspect a previous missed optimization became "latent" again (optimizing to a single BB again).  And we're running into one of the var-tracking limits (on expr depth I assume).

Interestingly with max-vartrack-size == 1 we still run into the compile-time
isse.

I suppose the data-flow problem doesn't converge.
Comment 8 Jakub Jelinek 2016-12-12 13:43:57 UTC 
For a single bb function I don't see how there could be a problem with converging during var-tracking.
I think the problem is elsewhere, during combine.
E.g. we have:
(debug_insn 3303 3302 3304 2 (var_location:QI __x (ne:QI (ne:QI (and:DI (reg:DI 118 [ cstore_186 ])
                (const_int 262144 [0x40000]))
            (const_int 0 [0]))
        (ne:QI (and:DI (reg:DI 118 [ cstore_186 ])
                (const_int 8388608 [0x800000]))
            (const_int 0 [0])))) -1
     (nil))
and combine calls propagate_for_debug that includes this and replaces
(reg:DI 118 [ cstore_186 ])
with
(if_then_else:DI (eq (reg:CCZ 17 flags)
        (const_int 0 [0]))
    (and:DI (reg:DI 109 [ _133 ])
        (const_int -4194305 [0xffffffffffbfffff]))
    (reg:DI 320))
so it has 4 pseudo references instead of 2.
After a while it is called again replacing
(reg:DI 109 [ _133 ])
with
(if_then_else:DI (eq (reg:CCZ 17 flags)
        (const_int 0 [0]))
    (and:DI (reg:DI 104 [ _103 ])
        (const_int -2097153 [0xffffffffffdfffff]))
    (reg:DI 356))
shortly it becomes:
(debug_insn 3303 3302 3304 2 (var_location:QI __x (ne:QI (ne:QI (and:DI (if_then_else:DI (eq (reg:CCZ 17 flags)
                        (const_int 0 [0]))
                    (and:DI (if_then_else:DI (eq (reg:CCZ 17 flags)
                                (const_int 0 [0]))
                            (and:DI (reg:DI 104 [ _103 ])
                                (const_int -2097153 [0xffffffffffdfffff]))
                            (reg:DI 356))
                        (const_int -4194305 [0xffffffffffbfffff]))
                    (ior:DI (if_then_else:DI (eq (reg:CCZ 17 flags)
                                (const_int 0 [0]))
                            (and:DI (reg:DI 104 [ _103 ])
                                (const_int -2097153 [0xffffffffffdfffff]))
                            (reg:DI 356))
                        (const_int 4194304 [0x400000])))
                (const_int 262144 [0x40000]))
            (const_int 0 [0]))
        (ne:QI (and:DI (if_then_else:DI (eq (reg:CCZ 17 flags)
                        (const_int 0 [0]))
                    (and:DI (if_then_else:DI (eq (reg:CCZ 17 flags)
                                (const_int 0 [0]))
                            (and:DI (reg:DI 104 [ _103 ])
                                (const_int -2097153 [0xffffffffffdfffff]))
                            (reg:DI 356))
                        (const_int -4194305 [0xffffffffffbfffff]))
                    (ior:DI (if_then_else:DI (eq (reg:CCZ 17 flags)
                                (const_int 0 [0]))
                            (and:DI (reg:DI 104 [ _103 ])
                                (const_int -2097153 [0xffffffffffdfffff]))
                            (reg:DI 356))
                        (const_int 4194304 [0x400000])))
                (const_int 8388608 [0x800000]))
            (const_int 0 [0])))) -1
     (nil))
and that grows further and further into a huge expression that is very expensive to handle e.g. in nonzero_bits, etc.
So, I wonder if we don't want something in propagate_for_debug if simplify_replace_fn_rtx returns already too complex expression to split it apart using debug temporaries into smaller parts (or another alternative is to reset the debug insn if it grows into a too complex expression).
In var-tracking.c we have also params that punt over certain expression complexity (--param max-vartrack-expr-depth=12), which is kind of an example that very complex expressions can be thrown away.
Or in cfgexpand.c avoid_deep_ter_for_debug is an example of function that splits too complex debug expressions (over hardcoded depth of TER).
Comment 9 Jakub Jelinek 2016-12-12 16:33:36 UTC 
Created attachment 40314 [details]
gcc7-pr77844.patch

Untested fix.
Comment 10 Jakub Jelinek 2016-12-14 19:40:37 UTC 
Author: jakub
Date: Wed Dec 14 19:40:05 2016
New Revision: 243662

URL: https://gcc.gnu.org/viewcvs?rev=243662&root=gcc&view=rev
Log:
	PR debug/77844
	* valtrack.c: Include rtl-iter.h.
	(struct rtx_subst_pair): Add insn field.
	(propagate_for_debug_subst): If pair->to contains at least 2
	regs, create a DEBUG_INSN with a debug temp before pair->insn
	and replace from with the debug temp instead of pair->to.
	(propagate_for_debug): Initialize p.insn.
	* combine.c (insn_uid_check): New inline function.
	(INSN_COST, LOG_LINKS): Use it instead of INSN_UID.
	(find_single_use, combine_instructions,
	cant_combine_insn_p, try_combine): Use NONDEBUG_INSN_P instead of
	INSN_P.

	* g++.dg/opt/pr77844.C: New test.

Added:
    trunk/gcc/testsuite/g++.dg/opt/pr77844.C
Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/combine.c
    trunk/gcc/testsuite/ChangeLog
    trunk/gcc/valtrack.c
Comment 11 Jakub Jelinek 2016-12-16 18:27:33 UTC 
Fixed on the trunk.
Comment 12 Jakub Jelinek 2017-10-11 08:36:00 UTC 
Fixed for 7+, 6.x doesn't exhibit this problem.
Comment 13 Martin Liška 2018-03-08 08:57:17 UTC 
Author: marxin
Date: Thu Mar  8 08:56:45 2018
New Revision: 258359

URL: https://gcc.gnu.org/viewcvs?rev=258359&root=gcc&view=rev
Log:
Backport r243662

2018-03-08  Martin Liska  <mliska@suse.cz>

	Backport from mainline
	2016-12-14  Jakub Jelinek  <jakub@redhat.com>

	PR debug/77844
	* valtrack.c: Include rtl-iter.h.
	(struct rtx_subst_pair): Add insn field.
	(propagate_for_debug_subst): If pair->to contains at least 2
	regs, create a DEBUG_INSN with a debug temp before pair->insn
	and replace from with the debug temp instead of pair->to.
	(propagate_for_debug): Initialize p.insn.
	* combine.c (insn_uid_check): New inline function.
	(INSN_COST, LOG_LINKS): Use it instead of INSN_UID.
	(find_single_use, combine_instructions,
	cant_combine_insn_p, try_combine): Use NONDEBUG_INSN_P instead of
	INSN_P.
2018-03-08  Martin Liska  <mliska@suse.cz>

	Backport from mainline
	2016-12-14  Jakub Jelinek  <jakub@redhat.com>

	PR debug/77844
	* g++.dg/opt/pr77844.C: New test.

Added:
    branches/gcc-6-branch/gcc/testsuite/g++.dg/opt/pr77844.C
Modified:
    branches/gcc-6-branch/gcc/ChangeLog
    branches/gcc-6-branch/gcc/combine.c
    branches/gcc-6-branch/gcc/testsuite/ChangeLog
    branches/gcc-6-branch/gcc/valtrack.c