Description:

Compiling an array with an initializer list takes memory roughly 140 times the size of the array. For example, and array of unsigned char with 1M entries (i.e., sizeof is 1MB), takes 140 MB to compile. An array of 5MB requires over 600MB of memory to compile. Below are some rough measurements based on the code,

 const unsigned char arr[] = {0,0,0, /*...NUM zeros here*/ };

Array_Size  /  GCC_Memory
1M          /  149M
2M          /  289M
3M          /  510M
4M          /  566M
5M          /  623M

At the above rate, GCC could require over 4GB of memory to compile a 20MB array. In general, GCC will exhaust system memory when compiling an array of size (Total_System_Memory/140).

How-To-Repeat:

/* This code takes roughly (140*NUM) bytes of memory to compile.  */
const unsigned char arr[] = {0,0, /*... NUM more values here*/};

The code above compiled a "C" suffers the same problem, but GCC's memory usage is slightly lower compared to C++, at 100 times the size of the array.

Fix:

There is a workaround, but it is undesirable. Instead of an array, use a string
literal. A string literal does not appear to have this memory usage or
scalability defect. Example:

/* BAD: */
const unsigned char arr[]={0xAB,0xCD,0xEF,...};

/* WORKAROUND: */
const unsigned char *arr=(unsigned char*)"\xAB\xCD\xEF...";


Keywords: memory-hog