(In reply to comment #0)

I forgot to add that I configured VICE with a following command:

CFLAGS='-flto -fuse-linker-plugin -O3 -fomit-frame-pointer -march=pentium4' CXXFLAGS='-O3 -fomit-frame-pointer -funroll-loops -march=pentium4 -flto -fuse-linker-plugin' ./configure --prefix=/home/misty/gcc --enable-gnomeui --enable-fullscreen; make

If you forget --enable-gnomeui, build will fail in my experience, so this is important.

On the link command-line add -r -nostdlib and start removing as many objects
and libraries from the command-line as possible (preserving the crash of course).
If static libraries remain, open them up and add/remove individual object
files.

For the smallest set of object files we need preprocessed source and the
command-lines to produce them.

Created attachment 21055 [details]
Preprocessed source files to reproduce this problem

I was able to reduce this to two (2) source files. Quite an improvement, original was more than hundred files :) The exact command I used to reproduce this bug is:

g++ -r -nostdlib -O3 -fomit-frame-pointer -funroll-loops -march=pentium4 -flto -fuse-linker-plugin sid.ii wave.ii

The .ii files were created with

g++ -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE=\"resid\" -DVERSION=\"0.16vice\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DSIZEOF_INT=4 -I. -I.     -O3 -fomit-frame-pointer -funroll-loops -march=pentium4 -flto -fuse-linker-plugin -MT sid.o -MD -MP -MF ".deps/sid.Tpo" -E sid.cc > sid.ii

g++ -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE=\"resid\" -DVERSION=\"0.16vice\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DSIZEOF_INT=4 -I. -I.     -O3 -fomit-frame-pointer -funroll-loops -march=pentium4 -flto -fuse-linker-plugin -MT wave.o -MD -MP -MF ".deps/wave.Tpo" -E wave.cc > wave.ii

In the meantime I also built gcc trunk revision 161601 and this also fails with:

lto1: internal compiler error: vector VEC(cgraph_node_ptr,base) index domain error, in lto_cgraph_encoder_deref at lto-cgraph.c:141

Confirmed.  Works without -fuse-linker-plugin.

Reduced testcases:

wave.ii
-------
typedef unsigned int reg8;
typedef unsigned int reg12;
class WaveformGenerator {
public:
    reg8 readOSC();
    inline reg12 output();
    reg8 waveform;
    reg12 output____();
};
inline reg12 WaveformGenerator::output() {
    switch (waveform) {
        default:
            return output____();
    }
}
reg8 WaveformGenerator::readOSC() {
    return output() >> 4;
}

sid.ii
------
typedef unsigned int reg8;
typedef unsigned int reg12;
typedef int sound_sample;
class WaveformGenerator {
public:
    reg8 readOSC();
    inline reg12 output();
    reg8 waveform;
    reg12 output____();
};
inline reg12 WaveformGenerator::output() {
    switch (waveform) {
        case 0x0:
            return output____();
    }
}
class Voice {
    inline sound_sample output();
    WaveformGenerator wave;
    friend class SID;
};
inline sound_sample Voice::output() {
    return wave.output();
};
class Filter {
public:
    void clock(sound_sample voice1, sound_sample voice2, sound_sample voice3);
};
class SID {
    void clock();
    Voice voice[3];
    Filter filter;
};
void SID::clock() {
  filter.clock(voice[0].output(), voice[1].output(), voice[2].output());
}

rguenther@murzim:/tmp> g++-4.5 -r -nostdlib -O2 -march=pentium4 -flto -fuse-linker-plugin -m32 sid.3.ii wave.3.ii
/usr/bin/gold: error: /tmp/ccbAKTWf.lto.o: incompatible target
collect2: ld returned 1 exit status
rguenther@murzim:/tmp> g++-4.5 -r -nostdlib -O2 -march=pentium4 -flto -fuse-linker-plugin -m32 wave.3.ii sid.3.ii
lto1: internal compiler error: Segmentation fault
Please submit a full bug report,
with preprocessed source if appropriate.
See <http://bugs.opensuse.org/> for instructions.
lto-wrapper: g++-4.5 returned 1 exit status
/usr/bin/gold: fatal error: lto-wrapper failed
collect2: ld returned 1 exit status

On trunk with the original testcase we fail with

/abuild/rguenther/trunk-g/gcc> ./xgcc -B. -r -nostdlib -O -march=pentium4 -flto -fuse-linker-plugin sid.ii wave.ii -m32 -B ../lto-plugin/.libs

lto1: out of memory allocating 8589934596 bytes after a total of 520192 bytes
lto-wrapper: ./xgcc returned 1 exit status
/usr/bin/gold: fatal error: lto-wrapper failed
collect2: ld returned 1 exit status

#4  0x00000000007bd6e1 in input_cgraph_1 (file_data=0x7ffff7ee2300, 
    ib=0x166a340) at /space/rguenther/src/svn/trunk/gcc/lto-cgraph.c:1257
1257          char *str = (char *)xmalloc (len + 1);
(gdb) l
1252
1253      /* Input toplevel asms.  */
1254      len = lto_input_uleb128 (ib);
1255      while (len)
1256        {
1257          char *str = (char *)xmalloc (len + 1);
1258          for (i = 0; i < len; ++i)
1259            str[i] = lto_input_1_unsigned (ib);
1260          cgraph_add_asm_node (build_string (len, str));
1261          free (str);
(gdb) p len
$1 = 8589934595

so somehow the output cgraph is corrupt.  Works fine without using the
linker-plugin on trunk, too.

This issue is now fixed in trunk. I have no obligation to push it for 4.5 series as I am now able to build whole VICE successfully with 4.6 so I am closing the ticket.