Re: trimming STL's memory pool

[Ulrich Drepper <drepper at redhat dot com>]

Paolo Carlini wrote:

> A good question. I don't have a clear cut answer. I can tell you that
> there are very good reasons *not* to return memory, i.e., you don't
> really know when it's *really* safe, until the end of the process.

One always knows when it is safe to return memory.  Just not whether
it's optimal for performance to do so.


> We
> battled with that quite a few times. Also, assuming your OS uses virtual
> memory, bloating the *virtual* memory usage doesn't seem such a *big*
> problem, frankly:

You fail to see the problem.  Just because the userland allocator in
libstdc++ currently doesn't use memory doesn't mean these memory blocks
don't require system resources.  To the contrary.  From the kernel's
point of view, every piece of memory which has ever been written to must
be preserved until it is returned via brk() or munmap(), depending on
how it was allocated.  And since this is not file backed memory it means
the memory allocates real system resources (either RAM of swap).  File
backed data (like program data or mmaped file data) can be store in the
filesystem and therefore doesn't allocate rare system resources, but
anonymous memory is different.  It is a big problem.

This is why multi-level memory allocation is bad, it makes the situation
even worse.  malloc itself also cache memory, in addition to any other
allocator.  Fortunately the usual malloc implementation is pretty well
tuned to free system resources and it has knobs the user can turn to
tweak this.

I was always of the opinion that the default allocator in libstdc++
shouldn't do anything special and just use malloc.  Provide special
allocators for situations where the user knows what s/he is doing and
can embed knowledge about the allocation needs in the program.  And
don't even think for a second to use brk() in the libstdc++ allocator:
this is doomed from the get go, there can only be one direct user of the
heap.

-- 
â Ulrich Drepper â Red Hat, Inc. â 444 Castro St â Mountain View, CA â

Attachment: signature.asc
Description: OpenPGP digital signature