maximum memory for gcj-compiled executable?

[Lars Andersen]

On Wed, 2003-04-02 at 21:43, Boehm, Hans wrote:
> I would be very interested in more information about an application that grows to 100MB, 
> if it never has more than 10MB of live data.  In particular, the GC log, with the GC compiled 
> without -DSILENT would be interesting.  That can theoretically happen,
> but I've never seen it, and it should be extremely unlikely.
> I believe the gcj collector is always compiled to never shrink the heap, 
> which is its default.  Thus an application with a high peak memory
> usage would continue to run with a large heap.  On some platforms the
> collector does have support to unmap entirely empty pages if they
> haven't been needed for a while.  This might solve this problem.
> (Entirely empty pages are usually much more common than you might 
> think.)  It has been tested less than the default configuration.  It
> should work, though you probably have to turn it on rather manually
> with -DUSE_MUNMAP.
> 
First of all, I don't claim that live data is sharply below 10 MB at all
time. I debug used/free heap at short intervals, and there is always a
lot free. This is a peculiar program : it recieves a http request once
every 30secs. (Not average, literally every 30secs). Plus, it sends out
a http request every 10 minutes. This might or might not start a short
period of activity. The program sleeps thru periods of inactivity.

The development on the heap goes like this :

eat 20 kb (30sec request)
eat 20 kb
eat 20 kb
...
this goes on for a while
...
eat 20 kb
free 3000kb ( the garbage collector kicks in)

this is just great, and how it is supposed to work. But sometimes a
little extra is allocated to the heap. This might be due to increased
activity. 

Clearly, when a heap is almost full, and the GC can free almost all of
it, why not a smaller heap, conserving memory?


Hey, would incremental garbage collecting do the trick, you think?

If the heap was full of non-freeable stuff, it would be a different
story. But this is not the case.

I realize that the current behaviour is desired for most apps. But Sun
included the -mx -ms options for a reason, don't you think?

I will make that GC log.

> Is there a reason to prefer command line arguments  to environment variables 
> for setting the heap size?  I would prefer the latter, both because
> it's already done that way in later GC versions, and because the code
> can be isolated to the GC.  (I think Sun compatibility is not an issue
> here.  The executable command line is different anyway, and I
> definitely don't want to impose a maximum heap size by default.)

Well I'm not partial about it as long as it works. Currently only
initial heap size works. Other people have suggested the option path, I
just wanted to move things along.

Why set a default max heap size. There isn't one now!
If the max heap size option is present, use it. Otherwise don't.

> There are also some known reasons that heap size is currently suboptimal, but these 
> shouldn't normally add more than 10MB or so to the heap:
> 
> 1) The root size is way too big.  This has been discussed here before, and I think we agree 
> on at least an approach for fixing it.  This causes the collector to
> grow the heap to much to compensate for the root scanning time.  
> (You can tune the collector to compensate, but then it runs slowly.)
> 
> 2) Under win32, it tends to scan system malloced memory as roots. 
>
>  This is a misfeature, since it can't be relied on.  My current version 
>  attempts to fix it, but the fix is heuristic, since I don't know of a
>  way to identify malloced memory sections.  (The fix works well on a
>  small toy example :-) .)  This affects only Windows.

Well, the worst cases all manifested themselves on win32...

This is an important matter, and I'm spending a lot of time getting to
the bottom of it. 

I suspect my program uses less than 2-3MB effective heap. Setting a max
size to 3MB might make the GC run wild, but if I'm wrong I would get a
outofmemory exception. In the current situation I don't know whether
there is stray objects on the heap, errors in the gcj or what.



I think focus should be shifted from "is there an error in gcj" and "the
GC behaves like it should" over to "do we need this feature" kind of
discussion.
And I find myself wanting that feature all the time.

BTW, any idea when that max heap environment setting makes it into gcj?


/Lars Andersen