Bug 85716 - No easy way for end-user to tell what GCC is doing when compilation is slow
Summary: No easy way for end-user to tell what GCC is doing when compilation is slow
Status: ASSIGNED
Alias: None
Product: gcc
Classification: Unclassified
Component: other (show other bugs)
Version: unknown
: P3 normal
Target Milestone: ---
Assignee: David Malcolm
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-05-09 15:14 UTC by David Malcolm
Modified: 2022-05-27 08:19 UTC (History)
4 users (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed: 2018-05-10 00:00:00


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description David Malcolm 2018-05-09 15:14:35 UTC
From #gcc on freenode:

<PlasmaHH_> Hi, I just refactored some of my code and now gcc won't finish compiling and grows in memory size (18gig so far after 30 minutes) and I would like to know what it is doing there. Likely something is going on with some template instantiations or so, is there any sane debugging option that will tell what it does on a somewhat high level while going along?
* PlasmaHH_ is now known as PlasmaHH
<dmalcolm> PlasmaHH: -ftime-report would probably be the first thing to try
<PlasmaHH> dmalcolm: those seem to output only something after they are done, which doesn't seem to finish. at least it doesn't output anything here.
<dmalcolm> hmmm
<dmalcolm> maybe -Q then
<PlasmaHH> dmalcolm: I was afraid you come up with that, its tons of information and all on one line ^^ which is why I was looking for a somewhat abstract level... but guess there isn't much then...
<dmalcolm> PlasmaHH: maybe there is something, and I'm just not thinking of it.  Seems to me like a valid thing to be able to "ask the compiler" about.
<PlasmaHH> dmalcolm: maybe that giant amount of (to me) unreadable output of -Q is just a consequence of something going wrong...
<dmalcolm> PlasmaHH: if there were linebreaks, would it be more readable?
<dmalcolm> (my job is improving GCC usability, so it's a serious question)
<PlasmaHH> dmalcolm: definetly. since it seems to output every function... I tried to split on ) but thats not too good for when there is a ) in the function signature or so
<dmalcolm> PlasmaHH: I think I'll open a BZ about this for GCC 9; it seems like there's room for improvement here
<PlasmaHH> most of the part (where things get wrong) seem to be really weird, probably functors being passed to functors ad inf. so if they were on one line I could much easier see that its indeed one big functor, at the moment its just a big mess :/
<PlasmaHH> taking things apart a bit the signature of one function is so far mentioned 184049 times in that -Q dump, possibly at various recursion depths....
<PlasmaHH> that doesnt sound good
<dmalcolm> perhaps a infinite recursion bug, or perhaps some kind of exponential template expansion?


As noted above, I'm capturing this in BZ as it seems like there's room for improvement here.

Am I missing something, or is there no "easy" way (for some definition of "easy") for an end-user to track what GCC is doing during a very long compile?

Some "brainstorm"-ish ideas:

Should -Q add newlines?
Maybe with timestamps?
Maybe a option to have some kind of timestamp-based logging, emitting a series of lines of the form:
  [TIMESTAMP] Doing something
perhaps piggy-backing off of the timevar hooks, so that if something within GCC goes off into the weeds, the user can see e.g. which function it's having trouble with.
Comment 1 David Malcolm 2018-05-09 15:31:51 UTC
Another idea: maybe reuse the hierarchical logging support from libgccjit?  (possibly with an option to add timestamps or somesuch).

See:
  https://gcc.gnu.org/onlinedocs/jit/internals/index.html#example-of-log-file
Comment 2 David Malcolm 2018-05-09 15:32:38 UTC
(or indeed, showing GGC usage)
Comment 3 David Malcolm 2018-05-09 18:18:25 UTC
More from #gcc on freenode:

<TemptorSent> dmalcolm - A flag to have gcc spit out what it's doing on long-runnign compiles would be immensely useful! I'm still trying to debug compile runs with checking=all running incredibly slowly, and haven't been able to bifurcate a particular test yet.
<TemptorSent> ...perhaps a flag to echo the checking in progress and statistics as it runs?
[...reference to this PR...]
<TemptorSent> I think the timestamp logging would be great, as well as a marker for transitions between compilation phases.
<dmalcolm> TemptorSent: OK if I paste this IRC log into that BZ, or do you want to add a comment to it?
<dmalcolm> also, do you need absolute timestamps, or is relative-to-start-of-cc1 ok?
<dmalcolm> (I was just going to use relative to start of cc1)
<dmalcolm> (or cc1plus, whatever)
<TemptorSent> Relative should be fine, although tagging the start time with the absolute time may be useful.
Comment 4 David Malcolm 2018-05-09 18:24:04 UTC
<TemptorSent> Also, I'm not sure if it's available, but if we could print the parent process and commandline passed at the beginning of an error report, and tag the output for the error report with the PIDs, it would make tracking things in parallel builds much easier.
Comment 5 Eric Gallager 2018-05-10 00:23:59 UTC
I think I remember someone suggesting a progress bar at one point; it involved hacking the pass manager... 

(confirmed)
Comment 6 Eric Gallager 2018-07-02 20:30:07 UTC
(In reply to Eric Gallager from comment #5)
> I think I remember someone suggesting a progress bar at one point; it
> involved hacking the pass manager... 
> 
> (confirmed)

Right, I found the thread (I think) I was thinking of:
https://gcc.gnu.org/ml/gcc-patches/2017-10/msg00957.html
Comment 7 Eric Gallager 2018-10-02 02:35:04 UTC
Changing status to ASSIGNED since there's a recent-enough assignee
Comment 8 Eric Gallager 2018-10-02 02:37:30 UTC
Changing status to ASSIGNED since there's a recent-enough assignee
(In reply to Eric Gallager from comment #7)
> Changing status to ASSIGNED since there's a recent-enough assignee

Also cc-ing Sandra since she wrote this:

(In reply to Eric Gallager from comment #6)
> 
> Right, I found the thread (I think) I was thinking of:
> https://gcc.gnu.org/ml/gcc-patches/2017-10/msg00957.html
Comment 9 sandra 2018-10-03 03:12:42 UTC
Just to clarify, I didn't have anything to do with proposing a "progress bar"; I just needed to know whether the split1 pass had run yet.  And I ended up solving that problem by using a property instead of a dynamic pass numbering thing.
Comment 10 Eric Gallager 2019-01-04 02:18:33 UTC
(In reply to sandra from comment #9)
> Just to clarify, I didn't have anything to do with proposing a "progress
> bar"; I just needed to know whether the split1 pass had run yet.  And I
> ended up solving that problem by using a property instead of a dynamic pass
> numbering thing.

Oh, maybe I was thinking of some other thread on the lists then?
Comment 11 Jakub Jelinek 2019-05-03 09:17:06 UTC
GCC 9.1 has been released.
Comment 12 Jakub Jelinek 2019-08-12 08:56:14 UTC
GCC 9.2 has been released.
Comment 13 Jakub Jelinek 2020-03-12 11:58:34 UTC
GCC 9.3.0 has been released, adjusting target milestone.
Comment 14 Richard Biener 2021-06-01 08:11:35 UTC
GCC 9.4 is being released, retargeting bugs to GCC 9.5.