Re: trimming STL's memory pool

[Paolo Carlini <pcarlini at suse dot de>]

Ulrich Drepper wrote:

>Paolo Carlini wrote:
>
>> good question. I don't have a clear cut answer. I can tell you that
>>there are very good reasons *not* to return memory, i.e., you don't
>>really know when it's *really* safe, until the end of the process.
>>    
>>
>One always knows when it is safe to return memory.  Just not whether
>it's optimal for performance to do so.
>  
>
Probably I should have described the problems better. I was referring to
issues related to the implementation of the memory allocator itself,
then |__cxa_atexit|, this sort of thing. Well, maybe it's just us ;) ...

>>We
>>battled with that quite a few times. Also, assuming your OS uses virtual
>>memory, bloating the *virtual* memory usage doesn't seem such a *big*
>>problem, frankly:
>>    
>>
>You fail to see the problem.  Just because the userland allocator in
>libstdc++ currently doesn't use memory doesn't mean these memory blocks
>don't require system resources.  To the contrary.  From the kernel's
>point of view, every piece of memory which has ever been written to must
>be preserved until it is returned via brk() or munmap(), depending on
>how it was allocated.  And since this is not file backed memory it means
>the memory allocates real system resources (either RAM of swap).  File
>backed data (like program data or mmaped file data) can be store in the
>filesystem and therefore doesn't allocate rare system resources, but
>anonymous memory is different.  It is a big problem.
>  
>
Ok, thanks for the explanation. Of course I know that swapping has a
cost, in principle. Still, not many among the users of "our" pool-based
allocators complained, so far. Most certainly you are aware of much more
demanding applications than I am.

>This is why multi-level memory allocation is bad, it makes the situation
>even worse.  malloc itself also cache memory, in addition to any other
>allocator.  Fortunately the usual malloc implementation is pretty well
>tuned to free system resources and it has knobs the user can turn to
>tweak this.
>
>I was always of the opinion that the default allocator in libstdc++
>shouldn't do anything special and just use malloc.  Provide special
>allocators for situations where the user knows what s/he is doing and
>can embed knowledge about the allocation needs in the program.  And
>don't even think for a second to use brk() in the libstdc++ allocator:
>this is doomed from the get go, there can only be one direct user of the
>heap.
>
Indeed, this is what is *already* going on by default since 3.4.0.

Paolo.