Heap fragmentation (Was: Debugging "Leaks" With Boehm-GC)

[Martin Egholm Nielsen]

> Thanks.
:-)

> It looks like the first dump was generated right after a GC, but there
> was some intervening allocation between the last GC and the second dump?
> I'm asking because it contains a bunch of unmarked objects, which is OK
> if they were newly allocated, but not otherwise..
Yes exactly - that's what triggering the GC...

> You were trying to allocate something larger than about 450K, the size
> of the largest free block?
Sounds correct! Although I would guess that I would request exactly that 
amount of memory, since I'm doing the same operations over and over 
again. But if a small chunk of that piece is being taken by another 
previous object, the 450K may just be a few bits too small.

> You might check the finalizable object statistics, and make sure that
> number is not continuously growing over time.  It seems fairly high to
> me, but I don't have good gcj statistics to compare it to.
Ehm :-) Where do I find this "finalizable object statistics"?

> This looks like a large object fragmentation problem, which is
> theoretically unavoidable without moving objects, and we sometimes can't
> do much better in practice :-).  As always, the huge static root size
> makes this worse by causing the collector to collect too infrequently
> with the default heuristic.
Bummer!

> Unfortunately, I think your options are mostly the ones that have
> already been mentioned:
> 
> - Increase GC_free_space_divisor.
> - Avoid allocating very large objects, if you can.  (Turning a large
> single dimensional array into a two dimensional one is likely to help.)
> - Don't declare finalizers unless you really need them.  (Not that
> you're necessarily doing that.)
> - Wait for someone to fix/help fix the root size issue.
> 
> Unless I'm missing something, or my interpretation above is incorrect, I
> don't see anything really broken.
I didn't think so, but it would be stupid not to check :-)

I guess I'll aim for the increased divisor, plus a reduction in the big 
allocations sizes - that's a parameter I can change - however, not 
without breaking my high level client-server protocol ;-]

BR,
  Martin