Re: meek question

[Corey Kosak <kosak at cs dot cmu dot edu>]

> #define xmalloc(size)	xxmalloc(size, __FILE__, __LINE__)
> 
> And rearrange the current xmalloc function into xxmalloc that takes the
> extra arguments and logs them, perhaps to stderr, perhaps to new file.
> Then you can sort through that data (there will be a lot) and find where
> the big allocations are happening.
> 
> With that, we can likely find the culprit relatively easily.
> 
> Does this sound like something that you would be willing to tackle?


Hello,

I did the xmalloc logging you suggested (which was a bit of a nightmare since
there are a surprisingly large number of nearly-identical definitions of
xmalloc(), plus most routines call free() rather than xfree() (xfree is needed
so we can match xmallocs to xfrees), plus the obstack routines call xmalloc
and free indirectly (through a pointer) so the macro trick doesn't help there,
plus we need to log realloc and calloc too..... BUT I won't bore you any more
with the details :-)

Anyway, after studying the log and doing a bunch of tracing, I think I was
able to isolate the bug to the haifa scheduler.  By adding a single printf to
stock egcs-980221, I can show you what I think is the root cause of the bug:

Here are the steps I performed on both a Linux-x86 box and an alpha-osf4.0
box:

- decompress egcs-core-980221 and egcs-g++-980221

- apply the (small) alpha patch discussed in 
  http://www.cygnus.com/ml/egcs/1998-Feb/1011.html
  (otherwise it doesn't build)

- run configure (surprisingly, alpha appears to default to --enable-haifa,
  whereas Linux does not.  Bug or feature?)

- add a single fprintf to haifa-sched.c at line 8703 using this patch:




--- haifa-sched.c~      Tue Feb 17 16:35:41 1998
+++ haifa-sched.c       Tue Mar  3 03:23:10 1998
@@ -8700,6 +8700,8 @@
     emit_note_after (NOTE_INSN_DELETED, basic_block_end[n_basic_blocks - 1]);
 
   /* Schedule every region in the subroutine */
+  fprintf(stderr, "HELLO: nr_regions=%d max_reg_num=%d\n",
+        (int)nr_regions, (int)max_reg_num());
   for (rgn = 0; rgn < nr_regions; rgn++)
     {
       schedule_region (rgn);




Then build and install

Notice the difference in output when I run gcc on alpha vs linux:

LINUX:

> final_hell_loco/bin/g++ -O2 big.cc
HELLO: nr_regions=1 max_reg_num=2222


ALPHA:

> limit datasize 100M
> final_hell_loco/bin/g++ -O2 big.cc
HELLO: nr_regions=1002 max_reg_num=7172
HELLO: nr_regions=1002 max_reg_num=7172
big.cc: In function `int main()':
big.cc:59: virtual memory exhausted


I don't understand the operation of the scheduler at all, especially with
respect to the fact that the HELLO line gets printed twice on the alpha.  But
I traced cc1plus with gdb for a while after the second HELLO.  The above code
shows that schedule_region is called 1002 times; which eventually calls
sched_analyze_insn:

#1  0x1201d317c in sched_analyze_insn (x=0x0, insn=0x1400d8658, 
    loop_notes=0x80) at haifa-sched.c:3853
#2  0x1201d3340 in sched_analyze (head=0x11feb54a0, tail=0x1400d8bb8)
    at haifa-sched.c:3913
#3  0x1201db954 in compute_block_backward_dependences (bb=0)
    at haifa-sched.c:7254
#4  0x1201dcc7c in schedule_region (rgn=0) at haifa-sched.c:7558

in sched_analyze_insn you see this loop:

     for (i = 0; i < maxreg; i++)
	/* reg_last_sets[r] is now a list of insns */
	reg_last_sets[i]
	  = gen_rtx_INSN_LIST (VOIDmode, insn, NULL_RTX);

gen_rtx_INSN_LIST appears to allocate 24 bytes on the obstack (on the alpha)
most of the time (it appears to do so "most" of the time, though possibly not
every time.  I didn't track this precisely.)

So given the outer loop, the inner loop, and the 24 bytes, you get

1002 x 7172 x 24 = 172M

which would appear to explain the excessive memory use.



Why this is different on Linux vs Alpha, and why Linux gets one HELLO and
Alpha gets two, and why the first HELLO doesn't appear to run out of memory
but the second does---these are all mysteries too complex for me.
Nevertheless, I hope this provides enough of a clue, and if there are
additional tests that would be useful for me to try, please let me know.