[Gc] PRINT_STATS kind, and free_space_divisor
Thu Nov 25 08:50:00 GMT 2004
I looked at the collector code some more. We're still missing a bit
of information here.
I assume you have GC_DUMP_REGULARLY set. This actually dumps
the heap state from GC_finish_collection just after marking, but
before finalization. Normally this doesn't matter much. But since
you have 8000 finalizable objects, those may reference a fair amount of
memory, which should get marked during the GC_finalize() call.
Temporarily adding a COND_DUMP; after that call would be interesting,
to see whether the live data changes.
A more compact way to get live data information would be to rebuild
the collector without -DSILENT.
Especially if the results after GC_finalize() don't look much different,
it would also be very interesting to call GC_dump() when
GC_expand_hp_inner() is invoked (with a suitable GC_printf0(...)
to identify it in the log, or by calling it from the debugger),
to see what the heap looks like then. Somehow the collector thinks
it's sufficiently full, and we seem to disagree.
Perhaps the latter would give us some insight as to why HBLKSIZE matters.
I think it really shouldn't. The heap snapshots in your log show many
entirely empty pages few large objects, and an entirely unfragmented
large block free lists. All of this suggests there is no fragmentation
problem at all, and HBLKSIZE should be basically irrelevant.
Reducing HBLKSIZE to 256 probably adds substantial space overhead for
block headers, which wouldn't show up in the statistics. Thus I would
avoid that. My guess is that 512 and up is OK.
> -----Original Message-----
> From: Rutger Ovidius [mailto:email@example.com]
> Sent: Tuesday, November 23, 2004 11:37 PM
> To: Boehm, Hans
> Cc: firstname.lastname@example.org; email@example.com
> Subject: Re: [Gc] PRINT_STATS kind, and free_space_divisor
> Tuesday, November 23, 2004, 12:09:53 PM, you wrote:
> BH> That's somewhat encouraging. At least that seems to be an
> BH> improvement.
> BH> Re: new regions.
> BH> Unfortunately, it's hard to figure out what those are. They have
> BH> type MEM_PRIVATE. It would be wonderful if we could discard those
> BH> as well, and trace only from MEM_IMAGE sections. I'm pretty sure
> BH> that dll initialized data is mapped as MEM_IMAGE. Does anyone
> BH> know how win32 handles uninitialized data?
> For fun I have skipped MEM_PRIVATE types as well (in the same way as
> your previous patch for MEM_MAPPED), and my app still works with a
> static root set staying constant at 2.5 megs (still big I guess, but
> better than 15).
> I have no concept of what is in MEM_PRIVATE, nor if skipping these
> will cause any problems on any other type of OS, nor if there are
> problems that I can't see (would all problems reveal themselves in a
> crash, or are there more insidious possibilities?) Is it ok to skip
> MEM_MAPPED and MEM_PRIVATE on linux? I've only tried XP thus far and I
> don't know what would occur on NT/2K or 95/ME.
> BH> My guess is that these regions result from memory
> allocation outside
> BH> the GC. One other question would be whether there is a
> leak outside the
> BH> GC, which would cause the root set to grow, which would cause the
> BH> GC heap to grow.
> I don't know what this would be. The app uses SWT, which is jni. But,
> I'm not sure how to monitor its mem allocation.
> BH> The GC tries to discard sections used by malloc. But it only
> BH> knows how to do this by periodically calling malloc, discarding
> BH> the region containing the resulting pointer, and the
> BH> deallocating the memory it got. This is a rather primitive
> BH> heuristic, and we know how to do much better under Linux.
> BH> Re: total heap size
> BH> Did you multiply the size by the number of marks? Objects of
> BH> size smaller than 4K are allocated on 4K pages. The printed
> BH> block list contains one entry for each such page.
> Some nmarks are 0, but I counted the byte_size once anyway in the
> total. The rest I have now multiplied by nmarks and still result in a
> total of ~3.5 megs, in a heap of 21 megs. This still seems excessive
> to me. Since most java objects are small, for fun I have tried making
> the HBLKSIZE = 512. This was much better. I've also tried 256
> and my heap
> is now 7 megs. Since there are more heap blocks (blocks = 22992, bytes
> = 7068672), I guess this places a heavier burden on the CPU during
> each collection, but are there any other problems with manually
> setting this?
> Most of my objects (~90%) seem to be less than 256 bytes. A few are
> 16k-70k and two are 580k & 680k. Is manually tweaking HBLKSIZE the
> best way to find the sweet spot? (reminds me of a certain file system)
> Rutger Ovidius
More information about the Java