[PATCH 1/5] Use MADV_DONTNEED for freeing in garbage collector

Jakub Jelinek jakub@redhat.com
Mon Oct 10 10:48:00 GMT 2011


On Mon, Oct 10, 2011 at 12:25:15PM +0200, Richard Guenther wrote:
> On Sun, Oct 9, 2011 at 9:55 PM, Andi Kleen <andi@firstfloor.org> wrote:
> > From: Andi Kleen <ak@linux.intel.com>
> >
> > Use the Linux MADV_DONTNEED call to unmap free pages in the garbage
> > collector.Then keep the unmapped pages in the free list. This avoid
> > excessive memory fragmentation on large LTO bulds, which can lead
> > to gcc bumping into the Linux vm_max_map limit per process.
> >
> > Based on a idea from Jakub.
> 
> Shouldn't we prefer still "mapped" pages when allocating?  Thus, keep
> the freepages list "sorted"?

I don't see why.  MADV_DONTNEED isn't perfect, what it does (on Linux)
is that it zaps the whole page range, which essentially brings it into
the exact same state as immediately after mmap.  Any touch of the
pages will result in a zeroed page being inserted into the page tables.

4 years ago there was a MADV_FREE proposal which behaved much better
(page was removed from page tables only when the kernel actually needed
them for something else, if the page wasn't needed and has been accessed
again by the application, it would still contain the old content (which
the app couldn't rely on, it could as well be cleared), but it would be much
cheaper in that case.  With MADV_FREE it would be actually preferrable
to pick the MADV_FREEd pages over picking up freshly munmapped but not yet
touched pages.

> With the new params to call release_pages less, how does this
> interact with using MADV_DONTNEED?  The only reason to delay
> MADV_DONTNEED is to avoid splitting huge-pages?  Which would

Not just that.  MADV_DONTNEED needs to flush the dirty pages from the page
tables and when they are touched again, they need to be cleared (or
pre-cleared pages inserted).  So, while MADV_DONTNEED is less expensive than
munmap + mmap, it is still not free.

> > 2011-10-08   Andi Kleen <ak@linux.intel.com>

Two space in between name and <.
> >
> >        PR other/50636
> >        * config.in, configure: Regenerate.

Please write each file on a separate line, and better below
* configure.ac line because of which they have been regenerated.

> >
> > +  /* Unmapped page? */
> > +  bool unmapped;
> > +

Not sure if unmapped is the best name of the flag here, because
it hasn't been unmapped, it just has been madvised.  Under unmap
most people would imagine munmap I'd say.

	Jakub



More information about the Gcc-patches mailing list