Re: trimming STL's memory pool

[Ben <b_plony at yahoo dot com>]

Hi

--- Paolo Carlini <pcarlini@suse.de> wrote:

> Ben wrote:
> 
> >As far as I know gcc's STL default
> >allocator uses memory pool to boost
> >its performance.
> >
> Actually, this is the case only of GCC 3.3.x, not
> GCC 3.4.x and 4.0.x.
> 

>From what I see this *is* the case with 3.4.x
( try the sample code below), I didn't try
it with 4.0.x though
 

> > Is there some way to force shrinking
> >of STL's memory pool and returning memory to the
> >system ? Sometimes your program needs a large
> amount
> >of STL containers only for short period and
> bloating
> >the process forewer is very annoying.
> >  
> >
> A good question. I don't have a clear cut answer. I
> can tell you that
> there are very good reasons *not* to return memory,
> i.e., you don't
> really know when it's *really* safe, until the end
> of the process. We
> battled with that quite a few times. Also, assuming
> your OS uses virtual
> memory, bloating the *virtual* memory usage doesn't
> seem such a *big*
> problem, frankly: basically, after a while, the
> unused memory pages get
> swapped out and never return to physical memory.
> Which kind of
> difficulties are you experiencing, exactly?

First of all when other processes need the
memory, OS (it is Linux in my case) should
swap unused pages of bloated process and this
takes time, delaying other processes. Then again
when this process with big STL pool needs to 
allocate ( for short time ) large amount of 
containers, OS will spend long time swapping in
pages of pool, because it has no idea that most
of them contain irrelevant garbage. So it is more
than desirable to give process some control over
STL pool. By the way regular malloc in glibc also
has similar caching policy and there is undocumented
function malloc_trim to force it to return memory to
the system.

> 
> Paolo.
> 

regards,
Ben

==========================================

You can wath memory usage of this
test at each stage and then continue
typing Ctrl-C

#include <iostream>
#include <cstdlib>
#include <signal.h>
#include <vector>
#include <unistd.h>

using namespace std ;

void sh(int)
{
    return ;
}

int main(int argc, char **argv)
{
    signal(SIGINT, sh);

    if( argc != 3 ) {
        cerr << "Usage: " << argv[0] 
       << " num_of_vectors vector_size_(in ints)\n";
        return 1;
    }

    int count = atoi(argv[1]);
    int sz = atoi(argv[2]);

    cout << "started\n";
    pause();
    

    vector<vector<int > > vp(count);
    for( unsigned i = 0; i < vp.size() ; ++i) {
        vp[i].resize(sz);
    }
    cout << "allocated\n";
    pause();

    vp.clear();
    cout << "freed\n";
    pause();

    return 0;
}



		
__________________________________ 
Yahoo! Mail 
Stay connected, organized, and protected. Take the tour: 
http://tour.mail.yahoo.com/mailtour.html