This is the mail archive of the
gcc-patches@gcc.gnu.org
mailing list for the GCC project.
Re: [PATCH 1/5] Use MADV_DONTNEED for freeing in garbage collector
On Mon, Oct 10, 2011 at 12:45 PM, Jakub Jelinek <jakub@redhat.com> wrote:
> On Mon, Oct 10, 2011 at 12:25:15PM +0200, Richard Guenther wrote:
>> On Sun, Oct 9, 2011 at 9:55 PM, Andi Kleen <andi@firstfloor.org> wrote:
>> > From: Andi Kleen <ak@linux.intel.com>
>> >
>> > Use the Linux MADV_DONTNEED call to unmap free pages in the garbage
>> > collector.Then keep the unmapped pages in the free list. This avoid
>> > excessive memory fragmentation on large LTO bulds, which can lead
>> > to gcc bumping into the Linux vm_max_map limit per process.
>> >
>> > Based on a idea from Jakub.
>>
>> Shouldn't we prefer still "mapped" pages when allocating? ?Thus, keep
>> the freepages list "sorted"?
>
> I don't see why. ?MADV_DONTNEED isn't perfect, what it does (on Linux)
> is that it zaps the whole page range, which essentially brings it into
> the exact same state as immediately after mmap. ?Any touch of the
> pages will result in a zeroed page being inserted into the page tables.
Which means we save the zeroing when allocating non-MADV_DONTNEEDed
pages first. And will be eventually able to unmap zapped pages.
> 4 years ago there was a MADV_FREE proposal which behaved much better
> (page was removed from page tables only when the kernel actually needed
> them for something else, if the page wasn't needed and has been accessed
> again by the application, it would still contain the old content (which
> the app couldn't rely on, it could as well be cleared), but it would be much
> cheaper in that case. ?With MADV_FREE it would be actually preferrable
> to pick the MADV_FREEd pages over picking up freshly munmapped but not yet
> touched pages.
>
>> With the new params to call release_pages less, how does this
>> interact with using MADV_DONTNEED? ?The only reason to delay
>> MADV_DONTNEED is to avoid splitting huge-pages? ?Which would
>
> Not just that. ?MADV_DONTNEED needs to flush the dirty pages from the page
> tables and when they are touched again, they need to be cleared (or
> pre-cleared pages inserted). ?So, while MADV_DONTNEED is less expensive than
> munmap + mmap, it is still not free.
But it's free at madvise time. munmap is "synchronous" at least (well,
when file-backed).
>> > 2011-10-08 ? Andi Kleen <ak@linux.intel.com>
>
> Two space in between name and <.
>> >
>> > ? ? ? ?PR other/50636
>> > ? ? ? ?* config.in, configure: Regenerate.
>
> Please write each file on a separate line, and better below
> * configure.ac line because of which they have been regenerated.
>
>> >
>> > + ?/* Unmapped page? */
>> > + ?bool unmapped;
>> > +
>
> Not sure if unmapped is the best name of the flag here, because
> it hasn't been unmapped, it just has been madvised. ?Under unmap
> most people would imagine munmap I'd say.
>
> ? ? ? ?Jakub
>