Re: High memory usage compiling large ‘xxd -i’ output

[relliott@umn.edu]

Hello,
Thanks for the explanation; I can see the issue now.

Ryan


> On Apr 11, 2020, at 9:16 PM, Xi Ruoyao <xry111@mengyan1223.wang> wrote:
> 
> On 2020-04-11 16:31 -0500, relliott--- via Gcc-help wrote:
>> Hello,
>> 
>> Thanks for your reply.  I guess I don’t understand why this requires so much
>> memory all at once.  Is it not possible to write to the object file as the
>> initializer is parsed?
> 
> A compiler doesn't write to object file.  It only writes to assembly file. 
> Modern compilers are designed multi-layer so the parser (frontend) should not
> output assembly (it's the job of the backend).  And, doing that so will miss
> optimizations.  A simple code:
> 
> const int b[] = {0, 1, 2};
> 
> int foo(void)
> {
>  int i, x = 0;
>  for (i = 0; i < 3; i++)
>    x += b[i];
>  return x;
> }
> 
> The body of "foo" will be optimized to "return 3;" at -O2.  If we wrote the
> array content to assembly file and discarded the value in memory, this
> optimization will be impossible.
> 
> "xxd" a binary file and then compile the result with a compiler is just like
> "converting a .jpg to .png then back to .jpg".  A smarter way:
> 
>  ld -r -b binary some_big_binary_blob.bin -o some_big_binary_blob.o
> 
> There will be symbols _binary_some_big_binary_blob_bin_start,
> _binary_some_big_binary_blob_end in some_big_binary_blob.o.  Then it's possible
> to access the content of the binary blob with:
> 
>  extern const char _binary_some_big_binary_blob_bin_start;
>  extern const char _binary_some_big_binary_blob_bin_end;
> 
>  int main(void)
>  {
>    const char *begin = &_binary_some_big_binary_blob_bin_start;
>    const char *end = &_binary_some_big_binary_blob_bin_end;
>    const char *p;
>    for (p = begin; p != end; p++)
>      play_with(*p);
>  }
> -- 
> Xi Ruoyao <xry111@mengyan1223.wang>
> School of Aerospace Science and Technology, Xidian University
>