Varray memory consumption strikes back

[Jan Hubicka]

> >  and why we didn't developed some stack
> > implementation that don't need moving data around at reallocation and
> > don't have more convenient API than obstacks.
> Well, if you want to go forth and do that, feel free.  It's
> nontrivial.
> 
> Note that with the new scheme I've started checking in, we don't
> have block local varrays anymore, but instead we have one single
> toplevel varray with markers within it to denote where data from
> the each block ends.
> 
> A nice side effect of that is we can look at other allocation schemes
> now since we've got a small number of toplevel objects with a well
> defined lifetime.  Hell, they could even be implemented with obstacks
> (one obstack per toplevel varray).

Yes, your current way of converting the varrays to huge ones makes me
perfectly happy and I think it is finally something that helps us to get
out of the problem.  I realize that I never really understood enought to
that code to figure out myself this sollution. My little mind always
stopped on the scheme of reusing varrays and I was still under belief
that there are nested arrays becuase you need some kind of random access
to the whole stack that is dificult to do with one big array.  So thanks
a lot for comming with this! I believe that it will make also the code
much easier to understand now.

Using obstacks for these is obviously cheaper than varrays so it might
be nice incremental step.  What I was thinking of is to simply wrap
obstack with something like OBSTACK_PUSH_TREE / OBSTACK_POP_TREE so the
api is more symetric to what varrays has that I think is much more
readable.

In the case you like this idea, I can simply do that once you are
finished with the current thread of removing nested arrays.

I am in a hurry now so I will return to rest of your email soon.

Honza