GCC Bugzilla Bug 29442

When compiling gcc-4.1.1, there is at least a serious slowdown, and on many
machines a failure, when insn-attrtab is compiled. There are several threads on
the Gentoo forums about this, some involving machines with 512M of memory. This
file seems to be automatically generated, and has become so large that one user
graphically describes it as a "swapfest". Compiling on a machine with the
specified Gentoo minimum of 64M is not practical.

Is it possible to split this file?

> Compiling on a machine with the
> specified Gentoo minimum of 64M is not practical.

HEHEHEHEHEHEHEHE. 64MB is very very small ammount of memory really.  Even the
PS3 has 256MB.  Oh and compiling GCC should not have a memory requirement so
gentoo is doing something wrong.

the "minimum" is a suggestion in our install docs ... it's like the "minimum"
you see on the side of boxed software in the store

the 64M has nothing to do with GCC

I think the pertinent part of this "bug" is:
> Is it possible to split this file?

  It took 6-8 hours to compile this file alone on my xbox, which only has 64m
of memory but should be considered a sufficiently modern machine.
  My understanding is that this is a generated file, containing information
about lots of different architectures.  Might it be possible to generate a file
per architecture, or per group of architectures?

  This is not a new problem, this file is what made me give up on Linux for the
Dreamcast (Which might not be considered a sufficiently modern machine.)

The problem is not only confined to "sufficiently modern machines". Some of us
have a conservationist sort of outlook, and are happy to find a use for an
older machine as a router, firewall, Kerberos server or whatever. Linux
supports this sort of thing well - a floppy firewall with no hard drive will
run just fine on an 8M machine, and a 486/33 will keep up with all that an
ordinary ADSL connection can transmit. With insn-attrtab excluded, gcc-4.1.1
compiles quickly and well on 52M, and quite possibly on smaller machines,
though I have no very recent experience to prove it. It seems a pity that a
single source file, and an automatically-generated one at that, should
essentially block an excellent compiler from a lot of smaller machines, new and
old, and cause enough of a slowdown even on a 512M machine to arouse concern.

If there is a rational reason for this file to be huge, well and good, but if
it is readily split that would seem like a worthwhile thing to do.

I have experienced the same problem.  I am compiling gcc on a machine with 32
mb ram.  Even with over 2 Gb swap it will work for hours and then finally crash
with an out of memory exception.  I have worked around the problem by compiling
gcc on another machine, and successfully resumed the  build.  (no word if it
worked yet).  Are there any patches that fix/ alleviate this problem?

As far as I can tell (from running cc1 in a debugger), the problem is not so
much the size of the file, but that it contains two large functions and GCC
leaks memory. After compiling the first large function, the process size is 886
megs (basic-block reordering alone being responsible for about 200 megs),
probably littered with pieces of garbage. That'll cause all following
allocations to become scattered thoughout memory and the swapfest begins if you
don't have lots of memory. 256 megs is not enough. The RTL passes in particular
seem to be affected and several df_* functions take ages to run.
I'm working on a patch to split insn-attrtab.c into more files.

Created an attachment (id=14420) [edit]
Reenable alloca() on non-GCC compilers

The memory fragmentation problem is to be caused by libiberty which disables
alloca() if you bootstrap with a non-GCC compiler[1] (which I do). Instead you
get a malloc() based replacment. The df_* functions use alloca() and are called
a lot.
With this patch on top of the next one, process size tops at around 185 MB
compared to the 900+ MB it reached before, at which point I gave up with 256 MB
of RAM.
Just wondering, which compiler are you using to bootstrap GCC?

[1] http://gcc.gnu.org/ml/gcc-patches/2001-03/msg00194.html

Created an attachment (id=14421) [edit]
split insn-attrtab.c into three files

Here's the patch to split insn-attrtab.c into smaller pieces. The result:
$ wc -l insn-*tab.c
  14566 insn-attrtab.c
  44454 insn-dfatab.c
  36815 insn-latencytab.c
  95835 total

Please tell me if it makes a difference.

(It is necessary to run autoheader and autoconf in the gcc directory after
applying the patches.)

I just tried with only the alloca patch, and despite the unsplit insn-attrtab.c
file, process size tops at just 205 MB. It looks like GCC 4.3.0 is in a much
better shape than GCC 4.1.1, so I'm letting go of the bug.

Just to clarify, the process sizes where taken when the stage1 compiler was
compiling the stage2 compiler.

What happened with the alloca patch? Looks like it was never committed.