RE: trimming STL's memory pool

["Emery Berger" <emery at cs dot umass dot edu>]

For answers to all these and other problems, see our paper to appear at
MSP (Feng & Berger, "A Locality-Improving Dynamic Memory Allocator"), on
my web page...

http://www.cs.umass.edu/~emery

The allocator described therein discards pages as suggested below (via
madvise) and manages memory via mmap rather than sbrk, allowing it to
return memory to the system from anywhere in the heap. The resulting
memory consumption can be a lot lower than for dlmalloc (the basis of
the GNU allocator).

Regards,
-- emery

> It would be nice if the allocator could somehow
> mark the free pages in the heap's holes to inform
> the OS, that it can throw them away, when they need
> to be swapped instead of writing them to the swap
> ( maybe something similar to dirty bit ) because the
> data in this pages is irrelevant. The next
> time the allocator will need to use these pages ,
> RAM frames will be allocated through regular minor
> page faults without disk I/O.
> 
> > This is why multi-level memory allocation is bad, it
> > makes the situation
> > even worse.  malloc itself also cache memory, in
> > addition to any other
> > allocator.  Fortunately the usual malloc
> > implementation is pretty well
> > tuned to free system resources and it has knobs the
> > user can turn to
> > tweak this.
> >
> 
> malloc has its own problems. If some piece of data
> remains at the top of the data segment ( no free
> top-most memory ) all the freed memory beneath it
> cannot be returned to the system.
>


--
Emery Berger
Assistant Professor
Dept. of Computer Science
University of Massachusetts, Amherst
www.cs.umass.edu/~emery
 

> -----Original Message-----
> From: Ben [mailto:b_plony@yahoo.com]
> Sent: Thursday, June 02, 2005 12:21 PM
> To: Ulrich Drepper
> Cc: libstdc++@gcc.gnu.org
> Subject: Re: trimming STL's memory pool
> 
> Hi
> 
> --- Ulrich Drepper <drepper@redhat.com> wrote:
> 
> > You fail to see the problem.  Just because the
> > userland allocator in
> > libstdc++ currently doesn't use memory doesn't mean
> > these memory blocks
> > don't require system resources.  To the contrary.
> > From the kernel's
> > point of view, every piece of memory which has ever
> > been written to must
> > be preserved until it is returned via brk() or
> > munmap(), depending on
> > how it was allocated.
> 
> It would be nice if the allocator could somehow
> mark the free pages in the heap's holes to inform
> the OS, that it can throw them away, when they need
> to be swapped instead of writing them to the swap
> ( maybe something similar to dirty bit ) because the
> data in this pages is irrelevant. The next
> time the allocator will need to use these pages ,
> RAM frames will be allocated through regular minor
> page faults without disk I/O.
> 
> > This is why multi-level memory allocation is bad, it
> > makes the situation
> > even worse.  malloc itself also cache memory, in
> > addition to any other
> > allocator.  Fortunately the usual malloc
> > implementation is pretty well
> > tuned to free system resources and it has knobs the
> > user can turn to
> > tweak this.
> >
> 
> malloc has its own problems. If some piece of data
> remains at the top of the data segment ( no free
> top-most memory ) all the freed memory beneath it
> cannot be returned to the system.
> 
> 
> 
> 
> 
> __________________________________
> Discover Yahoo!
> Find restaurants, movies, travel and more fun for the weekend. Check
it
> out!
> http://discover.yahoo.com/weekend.html