This is the mail archive of the gcc-patches@gcc.gnu.org mailing list for the GCC project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [PATCH] LTO streamer reorg - try to reduce WPA memory use



On 07/30/2014 11:41 AM, Richard Biener wrote:
On Wed, 30 Jul 2014, Richard Biener wrote:

On Wed, Jul 30, 2014 at 7:51 AM, Markus Trippelsdorf
<markus@trippelsdorf.de> wrote:
On 2014.07.29 at 15:10 +0200, Richard Biener wrote:
On Tue, 29 Jul 2014, Richard Biener wrote:

This re-organizes the LTO streamer to do compression transparently
in the data-streamer routines (and disables section compression
by defaulting to -flto-compression-level=0).  This avoids
keeping the whole uncompressed sections in memory, only retaining
the compressed ones.

The downside is that we lose compression of at least the string
parts (they are abusing the streaming interface quite awkwardly
and doing random-accesses with offsets into the uncompressed
section).  With a little bit of surgery we can get that back I
think (but we'd have to keep the uncompressed piece in memory
somewhere which means losing the memory use advantage).

Very lightly tested sofar (running lto.exp).  I'll try a LTO
bootstrap now.

I wonder what the change is on WPA memory use for larger
projects and what the effect on object file size is.
Updated patch passing LTO bootstrap (one warning fix) and
with a memory leak fixed.
Testing with Firefox is impossible at the moment because of PR61885.
One thing I've noticed (before the ICE) is that virtual memory usage is
very high:

Address            Kbytes      RSS    Dirty  Mode  Mapping
0000000000400000    16344     9084        0  r-x-- lto1
00000000013f6000       36       36       28  rw--- lto1
00000000013ff000     1072      276      276  rw---   [ anon ]
00000000034aa000 10154940  1540384  1540384  rw---   [ anon ]
00002acf04af2000      136      136        0  r-x-- ld-2.19.90.so
00002acf04b14000       88       88       88  rw---   [ anon ]
...
----------------  -------  -------  -------
total kB         12022060  3388396  3377708
Maybe there is still a memleak (just checked that LTOing int main() {}
doesn't leak).
Found it:

Index: gcc/lto-section-in.c
===================================================================
--- gcc/lto-section-in.c.orig   2014-07-30 12:40:27.950225826 +0200
+++ gcc/lto-section-in.c        2014-07-30 12:37:44.179237102 +0200
@@ -249,7 +249,7 @@ lto_destroy_simple_input_block (struct l
                                 struct lto_input_block *ib,
                                 const char *data, size_t len)
  {
-  free (ib);
+  delete ib;
    lto_free_section_data (file_data, section_type, NULL, data, len);
  }
Richard.
Hello,
   there's memory/CPU usage for the patch. for both, I used sync and drop_caches.

Url: https://drive.google.com/file/d/0B0pisUJ80pO1andOX19JMHV3LVE/edit?usp=sharing

Martin


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]