Bug 44066 - excessive memory usage when compiling array initializers.
Summary: excessive memory usage when compiling array initializers.
Status: RESOLVED DUPLICATE of bug 14179
Alias: None
Product: gcc
Classification: Unclassified
Component: c++ (show other bugs)
Version: 4.5.0
: P3 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
Depends on:
Reported: 2010-05-10 17:43 UTC by john
Modified: 2010-05-10 22:06 UTC (History)
11 users (show)

See Also:
Host: i686-pc-linux-gnu
Target: i686-pc-linux-gnu
Build: i686-pc-linux-gnu
Known to work:
Known to fail:
Last reconfirmed:


Note You need to log in before you can comment on or make changes to this bug.
Description john 2010-05-10 17:43:56 UTC

Compiling an array with an initializer list takes memory roughly 140 times the size of the array. For example, and array of unsigned char with 1M entries (i.e., sizeof is 1MB), takes 140 MB to compile. An array of 5MB requires over 600MB of memory to compile. Below are some rough measurements based on the code,

 const unsigned char arr[] = {0,0,0, /*...NUM zeros here*/ };

Array_Size  /  GCC_Memory
1M          /  149M
2M          /  289M
3M          /  510M
4M          /  566M
5M          /  623M

At the above rate, GCC could require over 4GB of memory to compile a 20MB array. In general, GCC will exhaust system memory when compiling an array of size (Total_System_Memory/140).


/* This code takes roughly (140*NUM) bytes of memory to compile.  */
const unsigned char arr[] = {0,0, /*... NUM more values here*/};

The code above compiled a "C" suffers the same problem, but GCC's memory usage is slightly lower compared to C++, at 100 times the size of the array.


There is a workaround, but it is undesirable. Instead of an array, use a string
literal. A string literal does not appear to have this memory usage or
scalability defect. Example:

/* BAD: */
const unsigned char arr[]={0xAB,0xCD,0xEF,...};

const unsigned char *arr=(unsigned char*)"\xAB\xCD\xEF...";

Keywords: memory-hog
Comment 1 Andrew Pinski 2010-05-10 22:06:36 UTC

*** This bug has been marked as a duplicate of 14179 ***