Bug 10060 - ICE (stack overflow) on huge file (300k lines) due to recursive behaviour of copy_rtx_if_shared, in emit_rtl.c
Summary: ICE (stack overflow) on huge file (300k lines) due to recursive behaviour of ...
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: middle-end (show other bugs)
Version: 3.2.1
: P3 normal
Target Milestone: 3.3.3
Assignee: Andrew Pinski
URL:
Keywords: ice-on-valid-code, patch
Depends on:
Blocks:
 
Reported: 2003-03-13 12:26 UTC by andreas
Modified: 2010-10-27 16:59 UTC (History)
4 users (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed: 2003-12-07 06:56:28


Attachments
pw.cc (308 bytes, text/plain)
2003-05-21 15:17 UTC, andreas
Details
patch to fix this (1.49 KB, patch)
2003-07-20 02:14 UTC, Andrew Pinski
Details | Diff
better patch based on RTH's comments (1.39 KB, patch)
2003-12-07 06:53 UTC, Andrew Pinski
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description andreas 2003-03-13 12:26:00 UTC
g++ reports:
> g++ -pipe -ggdb -o t t.cc
{standard input}: Assembler messages:
{standard input}:597943: Warning: end of file not at end of a line; newline inserted
{standard input}:599822: Error: expected comma after name `a9' in .size directive
g++: Internal error: Segmentation Fault (program cc1plus)
Please submit a full bug report.
See <URL:http://www.gnu.org/software/gcc/bugs.html> for instructions.

The dumped core has a size of 826761032 bytes.
> pstack core | wc
74822  748214 6808444
> pstack core | fgrep ggc_mark_rtx_children_1 | wc
74806  748060 6807318

Release:
g++ (GCC) 3.2.1

Environment:
config.guess: sparc-sun-solaris2.8
../gcc-3.2.1/configure --enable-threads --with-as=/opt/gcc321/bin/as --with-gnu-as

How-To-Repeat:
The attached file must be compiled and then executed - the resulting file (which is *LARGE*) must be compiled:
> g++ -o pw pw.cc
> ./pw 100000 >t.cc
> g++ -pipe -ggdb -o t t.cc
Comment 1 Giovanni Bajo 2003-05-05 10:52:02 UTC
State-Changed-From-To: open->feedback
State-Changed-Why: Hello Andreas,
    I have troubles reproducing this because it either hits 
    virtual memory exhausted or compiles correctly, depends
    on how many objects I make it generate.
    Would you please confirm that this is still a bug in 
    GCC 3.3 or 3.4, and provide a minimal number of objects
    that must be generated to trigger it? It seems that
    compilation time grows O(n^2), so putting a big number
    means hours of compilation (assuming infinite memory).
Comment 2 andreas 2003-05-06 11:58:02 UTC
From: <andreas@florath.net>
To: giovannibajo@libero.it, andreas@florath.net, gcc-bugs@gcc.gnu.org, gcc-prs@gcc.gnu.org, giovannibajo@libero.it, nobody@gcc.gnu.org, gcc-gnats@gcc.gnu.org
Cc:  
Subject: Re: Re: c++/10060: ICE on huge file (300k lines)
Date: Tue,  6 May 2003 11:58:02 +0200

 Hello!
 
 	I just downloaded the latest gcc snapshot (gcc version 3.3
 20030428 (prerelease)).  When trying to compile the 100000 version,
 the error changed to
 
 > g++ -pipe -ggdb -o t t.cc
 {standard input}: Assembler messages:
 {standard input}:597943: Warning: end of file not at end of a line;
 newline inserted
 {standard input}:599806: Error: unknown pseudo-op: `.g'
 g++: Internal error: Segmentation Fault (program cc1plus)
 Please submit a full bug report.
 See <URL:http://gcc.gnu.org/bugs.html> for instructions.
 
 I'm now trying to figure out the smallest number when this error
 occurs: it's somewhat between 65536 (which compiles) and 100000 (which
 failes).  Please give me some time, because it takes some hours to
 compile.
 
 If you need some additional information (assembler output, core file
 :-) ...) feel free to ask. [Also one step I want to do is to check the
 size of the assembler output - maybe a problem with large files
 >4G????]
 
 	Best Regards
 
 	Andreas Florath
 	andreas@florath.net
 
 
 http://gcc.gnu.org/cgi-bin/gnatsweb.pl?cmd=view%20audit-trail&database=gcc&pr=10060
Comment 3 Andrew Pinski 2003-05-09 09:01:54 UTC
From: Andrew Pinski <pinskia@physics.uc.edu>
To: <andreas@florath.net>
Cc: Andrew Pinski <pinskia@physics.uc.edu>, gcc-gnats@gcc.gnu.org,
   gcc-bugs@gcc.gnu.org, nobody@gcc.gnu.org, gcc-prs@gcc.gnu.org,
   giovannibajo@libero.it
Subject: Re: c++/10060: ICE on huge file (300k lines)
Date: Fri, 9 May 2003 09:01:54 -0400

 This looks like a stack overflow, a temporary fix is to up the limit on  
 the stacksize.
 The real fix is to change copy_rtx_if_shared to be non-recursive.
 
 
 Thanks,
 Andrew Pinski
 
 
 On Friday, May 9, 2003, at 08:56 US/Eastern, <andreas@florath.net>  
 wrote:
 
 >
 > Hello!
 >
 >
 >         The numbers are 74799 (which works) and 74800 (which failes).
 >
 >         Maybe one additional hint: I had a look at the stack from the
 > core from 88511. It says:
 >
 >  00281958 ggc_alloc (55, 0, 0, 550404, 0, 0)
 >  00129f98 copy_rtx_if_shared (0, 17527728, 0, 33a5c8, 2, 55) + 170
 >  00129f10 copy_rtx_if_shared (17527710, 17527b60, 0, 1, 33a5c8, 4c) +  
 > e8
 >  00129f10 copy_rtx_if_shared (0, 17527f98, 0, 0, 33a5c8, 3) + e8
 >  00129f10 copy_rtx_if_shared (0, 175303d8, 0, 0, 33a5c8, 3) + e8
 >  00129f10 copy_rtx_if_shared (0, 17530810, 0, 0, 33a5c8, 3) + e8
 > ***
 > [there are 74816 lines with copy_rtx_if_shared only with different
 > second argument.]
 > ***
 >  00129f10 copy_rtx_if_shared (0, 33e82498, 0, 0, 33a5c8, 3) + e8
 >  00129f10 copy_rtx_if_shared (0, 33e828d0, 0, 0, 33a5c8, 3) + e8
 >  00129f10 copy_rtx_if_shared (0, 33e82d08, 0, 0, 33a5c8, 3) + e8
 >  00129f10 copy_rtx_if_shared (0, 3, 0, 0, 33a5c8, 3) + e8
 >  00129f10 copy_rtx_if_shared (0, 2, 362cee40, 0, 33a5c8, 3) + e8
 >  00129948 unshare_all_rtl (a895e80, 18b2a50, 18b2a50, 18b2a50,  
 > 11df4a80,
 > 1906e00) + 6c
 >  002635f0 rest_of_compilation (a895e80, 40d4d, 3d2000, 3d2400, dbe04,
 > 3b9000) + 478
 >  000befc4 genrtl_finish_function (a895e80, a895e80, 0, 1906e00, 0,
 > 40d4d) + 68
 >  000bec60 expand_body (a895e80, a8aca80, a6, 0, 0, 0) + 230
 >  00094868 yyparse  (3f1c08, ffbee710, 3e5000, 0, 31e000, ffbee71a) +  
 > 9bc
 >  000dc080 c_common_parse_file (0, ff33fa9c, 0, 3d2b38, dc048, 3b9000) +
 > 38
 >  00262bdc compile_file (32e058, ffbeeb20, 1, 0, 0, 393398) + 34
 >  00267fa8 do_compile (3ebc00, ffbeead4, 0, 0, 0, 0) + 84
 >  0026806c toplev_main (15, ffbeead4, ffbeeb2c, 3c8680, 0, 0) + 5c
 >  0003d20c _start   (0, 0, 0, 0, 0, 0) + 5c
 >
 >
 >         If I can help you (sending some file, try a patch, ...), just
 > drop me a mail.
 >
 >
 >         Best regards
 >
 >         Andreas Florath
 >         andreas@florath.net
 >
 >
 > http://gcc.gnu.org/cgi-bin/gnatsweb.pl?cmd=view%20audit- 
 > trail&database=gcc&pr=10060
 >
 >
 

Comment 4 Andrew Pinski 2003-05-09 12:31:50 UTC
From: Andrew Pinski <pinskia@physics.uc.edu>
To: "Giovanni Bajo" <giovannibajo@libero.it>
Cc: Andrew Pinski <pinskia@physics.uc.edu>, <andreas@florath.net>,
   <gcc-gnats@gcc.gnu.org>, <gcc-bugs@gcc.gnu.org>, <nobody@gcc.gnu.org>,
   <gcc-prs@gcc.gnu.org>
Subject: Re: c++/10060: ICE on huge file (300k lines)
Date: Fri, 9 May 2003 12:31:50 -0400

 That seems like a good idea, since copy_rtx_if_shared is in emit-rtl.c.
 
 
 Thanks,
 Andrew Pinski
 
 
 On Friday, May 9, 2003, at 09:10 US/Eastern, Giovanni Bajo wrote:
 
 > Andrew Pinski <pinskia@physics.uc.edu> wrote:
 >
 >>>  00281958 ggc_alloc (55, 0, 0, 550404, 0, 0)
 >>>  00129f98 copy_rtx_if_shared (0, 17527728, 0, 33a5c8, 2, 55) + 170
 >>>  00129f10 copy_rtx_if_shared (17527710, 17527b60, 0, 1, 33a5c8, 4c) +
 >>> e8
 >>>  00129f10 copy_rtx_if_shared (0, 17527f98, 0, 0, 33a5c8, 3) + e8
 >>>  00129f10 copy_rtx_if_shared (0, 175303d8, 0, 0, 33a5c8, 3) + e8
 >>>  00129f10 copy_rtx_if_shared (0, 17530810, 0, 0, 33a5c8, 3) + e8
 >
 >> This looks like a stack overflow, a temporary fix is to up the limit 
 >> on
 >> the stacksize.
 >> The real fix is to change copy_rtx_if_shared to be non-recursive.
 >
 > I'm not confident with internals of GCC, but I guess this bug should be
 > moved to the "middle-end" section. Do you agree, Andrew?
 >
 > Giovanni Bajo
 >
 >
 >
 
Comment 5 andreas 2003-05-09 14:56:02 UTC
From: <andreas@florath.net>
To: gcc-gnats@gcc.gnu.org, gcc-bugs@gcc.gnu.org, nobody@gcc.gnu.org, gcc-prs@gcc.gnu.org, giovannibajo@libero.it, andreas@florath.net
Cc:  
Subject: Re: c++/10060: ICE on huge file (300k lines)
Date: Fri,  9 May 2003 14:56:02 +0200

 Hello!
 
 
         The numbers are 74799 (which works) and 74800 (which failes). 
 
         Maybe one additional hint: I had a look at the stack from the
 core from 88511. It says: 
 
  00281958 ggc_alloc (55, 0, 0, 550404, 0, 0)
  00129f98 copy_rtx_if_shared (0, 17527728, 0, 33a5c8, 2, 55) + 170
  00129f10 copy_rtx_if_shared (17527710, 17527b60, 0, 1, 33a5c8, 4c) + e8
  00129f10 copy_rtx_if_shared (0, 17527f98, 0, 0, 33a5c8, 3) + e8
  00129f10 copy_rtx_if_shared (0, 175303d8, 0, 0, 33a5c8, 3) + e8
  00129f10 copy_rtx_if_shared (0, 17530810, 0, 0, 33a5c8, 3) + e8
 ***
 [there are 74816 lines with copy_rtx_if_shared only with different
 second argument.]
 ***
  00129f10 copy_rtx_if_shared (0, 33e82498, 0, 0, 33a5c8, 3) + e8
  00129f10 copy_rtx_if_shared (0, 33e828d0, 0, 0, 33a5c8, 3) + e8
  00129f10 copy_rtx_if_shared (0, 33e82d08, 0, 0, 33a5c8, 3) + e8
  00129f10 copy_rtx_if_shared (0, 3, 0, 0, 33a5c8, 3) + e8
  00129f10 copy_rtx_if_shared (0, 2, 362cee40, 0, 33a5c8, 3) + e8
  00129948 unshare_all_rtl (a895e80, 18b2a50, 18b2a50, 18b2a50, 11df4a80,
 1906e00) + 6c
  002635f0 rest_of_compilation (a895e80, 40d4d, 3d2000, 3d2400, dbe04,
 3b9000) + 478
  000befc4 genrtl_finish_function (a895e80, a895e80, 0, 1906e00, 0,
 40d4d) + 68
  000bec60 expand_body (a895e80, a8aca80, a6, 0, 0, 0) + 230
  00094868 yyparse  (3f1c08, ffbee710, 3e5000, 0, 31e000, ffbee71a) + 9bc
  000dc080 c_common_parse_file (0, ff33fa9c, 0, 3d2b38, dc048, 3b9000) +
 38
  00262bdc compile_file (32e058, ffbeeb20, 1, 0, 0, 393398) + 34
  00267fa8 do_compile (3ebc00, ffbeead4, 0, 0, 0, 0) + 84
  0026806c toplev_main (15, ffbeead4, ffbeeb2c, 3c8680, 0, 0) + 5c
  0003d20c _start   (0, 0, 0, 0, 0, 0) + 5c
 
 
         If I can help you (sending some file, try a patch, ...), just
 drop me a mail.
 
 
         Best regards
 
         Andreas Florath
         andreas@florath.net
 
 
 http://gcc.gnu.org/cgi-bin/gnatsweb.pl?cmd=view%20audit-trail&database=gcc&pr=10060

Comment 6 Giovanni Bajo 2003-05-09 15:10:15 UTC
From: "Giovanni Bajo" <giovannibajo@libero.it>
To: <andreas@florath.net>,
	"Andrew Pinski" <pinskia@physics.uc.edu>
Cc: "Andrew Pinski" <pinskia@physics.uc.edu>,
	<gcc-gnats@gcc.gnu.org>,
	<gcc-bugs@gcc.gnu.org>,
	<nobody@gcc.gnu.org>,
	<gcc-prs@gcc.gnu.org>
Subject: Re: c++/10060: ICE on huge file (300k lines)
Date: Fri, 9 May 2003 15:10:15 +0200

 Andrew Pinski <pinskia@physics.uc.edu> wrote:
 
 >>  00281958 ggc_alloc (55, 0, 0, 550404, 0, 0)
 >>  00129f98 copy_rtx_if_shared (0, 17527728, 0, 33a5c8, 2, 55) + 170
 >>  00129f10 copy_rtx_if_shared (17527710, 17527b60, 0, 1, 33a5c8, 4c) +
 >> e8
 >>  00129f10 copy_rtx_if_shared (0, 17527f98, 0, 0, 33a5c8, 3) + e8
 >>  00129f10 copy_rtx_if_shared (0, 175303d8, 0, 0, 33a5c8, 3) + e8
 >>  00129f10 copy_rtx_if_shared (0, 17530810, 0, 0, 33a5c8, 3) + e8
 
 > This looks like a stack overflow, a temporary fix is to up the limit on
 > the stacksize.
 > The real fix is to change copy_rtx_if_shared to be non-recursive.
 
 I'm not confident with internals of GCC, but I guess this bug should be
 moved to the "middle-end" section. Do you agree, Andrew?
 
 Giovanni Bajo
 

Comment 7 Giovanni Bajo 2003-05-09 16:59:28 UTC
State-Changed-From-To: feedback->analyzed
State-Changed-Why: Feedback received, and further analysys based on stack trace
    done by Andrew. Moved the PR into the middle-end category,
    since that's where it belongs to. Updated the synopsis.
Comment 8 Andrew Pinski 2003-07-10 15:14:33 UTC
I working on a fix for the stack overflow in copy_rtx_if_shared (and one in copy_rtx).
Comment 9 Andrew Pinski 2003-07-10 23:24:24 UTC
I now have a fix which works with a stacksize of 512K which is the default on Mac OS X.  
The fix is to optimize the common case in most of the functions which walk the rtx, I will 
be posting when my copyright assignment goes through and I get cvs access.
Comment 10 Andrew Pinski 2003-07-10 23:30:30 UTC
I should say that I have only tested it with 10000, not 100000 which is what is listed in the 
bug but it should work for that case, I will test it but it is slow at compiling this test.  I have 
already filed a different bug describing one of the slowness that this file gets.
Comment 11 Andrew Pinski 2003-07-11 02:27:05 UTC
Okay I can confirm this fixes the 100000 case (and most likely higher ones too but I do 
not have the time to test them) but it is slow and memory hungry test case: 850M.
Comment 12 Andrew Pinski 2003-07-11 23:58:11 UTC
I am marking this as compile-time-hog and memory-hog because of my analysis after 
fixed the bug (well not quite, there were bugs in my patch, I could not bootstrap, will 
investiagate). Another place where most of the time is spent is in regclass.
Comment 13 Andrew Pinski 2003-07-13 18:26:45 UTC
I found my problem, it was just a typo.  I submitted the patch to gcc-patches.
Comment 14 Mark Mitchell 2003-07-20 01:14:58 UTC
Andrew --

Would you please attach the patch (or a link to gcc-patches message) to the PR?

Thanks,

-- Mark
Comment 15 Andrew Pinski 2003-07-20 02:14:53 UTC
Created attachment 4440 [details]
patch to fix this

PR middle-end/10060
	* emit-rtl.c (copy_rtx_if_shared): Split out into ...
	(copy_rtx_if_shared_1): here and add common case of "ee".
	(reset_used_flags): optimize common case of "ee". 

Submited: <http://gcc.gnu.org/ml/gcc-patches/2003-07/msg01310.html>
Comment 16 Andrew Pinski 2003-11-10 07:40:38 UTC
I will update the patch soon.
Comment 17 Andrew Pinski 2003-11-18 07:00:37 UTC
It is a little better on the tree-ssa because it removes some code before changing to rtl which is 
where this problem is.
Comment 18 Andrew Pinski 2003-12-07 06:53:52 UTC
Created attachment 5291 [details]
better patch based on RTH's comments

Need to think up a changelog still.
Comment 19 Andrew Pinski 2003-12-07 06:56:27 UTC
Will submit the new patch when I get sober :)
Really this is only a ice-on-valid-code as the slowness and memory hog are different bugs.
Comment 20 Andrew Pinski 2003-12-07 23:41:35 UTC
Patch Here: <http://gcc.gnu.org/ml/gcc-patches/2003-12/msg00723.html>.
Comment 21 CVS Commits 2003-12-08 17:30:59 UTC
Subject: Bug 10060

CVSROOT:	/cvs/gcc
Module name:	gcc
Changes by:	pinskia@gcc.gnu.org	2003-12-08 17:30:47

Modified files:
	gcc            : ChangeLog emit-rtl.c 

Log message:
	2003-12-08  Andrew Pinski  <pinskia@physics.uc.edu>
	
	PR middle-end/10060
	* emit-rtl.c (copy_rtx_if_shared): Split out into ...
	(copy_rtx_if_shared_1): here and optimize the last one
	in the sequence into tail-recursion.
	(reset_used_flags): Optimize the last one
	in the sequence into tail-recursion.

Patches:
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ChangeLog.diff?cvsroot=gcc&r1=2.1958&r2=2.1959
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/emit-rtl.c.diff?cvsroot=gcc&r1=1.355&r2=1.356

Comment 22 CVS Commits 2003-12-08 17:37:04 UTC
Subject: Bug 10060

CVSROOT:	/cvs/gcc
Module name:	gcc
Branch: 	gcc-3_3-branch
Changes by:	pinskia@gcc.gnu.org	2003-12-08 17:37:00

Modified files:
	gcc            : ChangeLog emit-rtl.c 

Log message:
	2003-12-08  Andrew Pinski  <pinskia@physics.uc.edu>
	
	PR middle-end/10060
	* emit-rtl.c (copy_rtx_if_shared): Split out into ...
	(copy_rtx_if_shared_1): here and optimize the last one
	in the sequence into tail-recursion.
	(reset_used_flags): Optimize the last one
	in the sequence into tail-recursion.

Patches:
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/ChangeLog.diff?cvsroot=gcc&only_with_tag=gcc-3_3-branch&r1=1.16114.2.830&r2=1.16114.2.831
http://gcc.gnu.org/cgi-bin/cvsweb.cgi/gcc/gcc/emit-rtl.c.diff?cvsroot=gcc&only_with_tag=gcc-3_3-branch&r1=1.303.2.2&r2=1.303.2.3

Comment 23 Andrew Pinski 2003-12-08 17:37:38 UTC
Fixed for 3.3.3 and 3.4.
Comment 24 Andrew Pinski 2003-12-29 21:41:54 UTC
*** Bug 10349 has been marked as a duplicate of this bug. ***