This is the mail archive of the
gcc-patches@gcc.gnu.org
mailing list for the GCC project.
Re: [PATCH] LTO streamer reorg - try to reduce WPA memory use
- From: Richard Biener <richard dot guenther at gmail dot com>
- To: Markus Trippelsdorf <markus at trippelsdorf dot de>
- Cc: Richard Biener <rguenther at suse dot de>, GCC Patches <gcc-patches at gcc dot gnu dot org>, Jan Hubicka <hubicka at ucw dot cz>
- Date: Wed, 30 Jul 2014 10:31:40 +0200
- Subject: Re: [PATCH] LTO streamer reorg - try to reduce WPA memory use
- Authentication-results: sourceware.org; auth=none
- References: <alpine dot LSU dot 2 dot 11 dot 1407291309570 dot 3531 at zhemvz dot fhfr dot qr> <alpine dot LSU dot 2 dot 11 dot 1407291508480 dot 3531 at zhemvz dot fhfr dot qr> <20140730055113 dot GE22904 at x4>
On Wed, Jul 30, 2014 at 7:51 AM, Markus Trippelsdorf
<markus@trippelsdorf.de> wrote:
> On 2014.07.29 at 15:10 +0200, Richard Biener wrote:
>> On Tue, 29 Jul 2014, Richard Biener wrote:
>>
>> >
>> > This re-organizes the LTO streamer to do compression transparently
>> > in the data-streamer routines (and disables section compression
>> > by defaulting to -flto-compression-level=0). This avoids
>> > keeping the whole uncompressed sections in memory, only retaining
>> > the compressed ones.
>> >
>> > The downside is that we lose compression of at least the string
>> > parts (they are abusing the streaming interface quite awkwardly
>> > and doing random-accesses with offsets into the uncompressed
>> > section). With a little bit of surgery we can get that back I
>> > think (but we'd have to keep the uncompressed piece in memory
>> > somewhere which means losing the memory use advantage).
>> >
>> > Very lightly tested sofar (running lto.exp). I'll try a LTO
>> > bootstrap now.
>> >
>> > I wonder what the change is on WPA memory use for larger
>> > projects and what the effect on object file size is.
>>
>> Updated patch passing LTO bootstrap (one warning fix) and
>> with a memory leak fixed.
>
> Testing with Firefox is impossible at the moment because of PR61885.
> One thing I've noticed (before the ICE) is that virtual memory usage is
> very high:
>
> Address Kbytes RSS Dirty Mode Mapping
> 0000000000400000 16344 9084 0 r-x-- lto1
> 00000000013f6000 36 36 28 rw--- lto1
> 00000000013ff000 1072 276 276 rw--- [ anon ]
> 00000000034aa000 10154940 1540384 1540384 rw--- [ anon ]
> 00002acf04af2000 136 136 0 r-x-- ld-2.19.90.so
> 00002acf04b14000 88 88 88 rw--- [ anon ]
> ...
> ---------------- ------- ------- -------
> total kB 12022060 3388396 3377708
Maybe there is still a memleak (just checked that LTOing int main() {}
doesn't leak).
Otherwise I forgot to enable compression at all ;)
Richard.
> --
> Markus