Bug 56533 - Linker problem on avr with lto and main function inside archive
Linker problem on avr with lto and main function inside archive
Status: RESOLVED INVALID
Product: gcc
Classification: Unclassified
Component: target
4.7.2
: P3 normal
: ---
Assigned To: Not yet assigned to anyone
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2013-03-05 12:33 UTC by Matthijs Kooijman
Modified: 2013-03-05 14:40 UTC (History)
0 users

See Also:
Host:
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Matthijs Kooijman 2013-03-05 12:33:17 UTC
When trying to add lto to my Arduino program, it stopped compiling complaining about missing symbols. I've managed to reduce the problem to below minimal example. Note that removing anything from below example makes the problem disappear. In particular, the problem disappears when:
 * any of the linker options is removed: -mmcu=atmega328p -Os -flto -fwhole-program
 * the -flto compiler option is removed
 * using normal gcc (amd64) instead of avr-gcc
 * linking main.o instead of main.a
 * declaring realmain as externally_visible in realmain.c

Note that in this example, the actual main() function is inside an archive, which is probably the reason for this bug / problem.



$ avr-gcc --version
	avr-gcc (GCC) 4.7.2
	Copyright (C) 2012 Free Software Foundation, Inc.
	This is free software; see the source for copying conditions.  There is NO
	warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ avr-ld --version
	GNU ld (GNU Binutils) 2.20.1.20100303
	Copyright 2009 Free Software Foundation, Inc.
	This program is free software; you may redistribute it under the terms of
	the GNU General Public License version 3 or (at your option) a later version.
	This program has absolutely no warranty.

$ cat main.c
	int realmain(void);

	int main(void)
	{
		return realmain();
	}

$ cat realmain.c
	int realmain(void) {
	}

$ cat do
	#!/bin/sh
	set -x

	rm -f main.a

	/usr/bin/avr-gcc -c main.c -o main.o
	/usr/bin/avr-ar rcs main.a  main.o
	/usr/bin/avr-gcc -c -flto realmain.c -o realmain.o
	/usr/bin/avr-gcc -mmcu=atmega328p -Os -flto -fwhole-program realmain.o main.a

$ ./do
	+ rm -f main.a
	+ /usr/bin/avr-gcc -c main.c -o main.o
	+ /usr/bin/avr-ar rcs main.a main.o
	+ /usr/bin/avr-gcc -c -flto realmain.c -o realmain.o
	+ /usr/bin/avr-gcc -mmcu=atmega328p -Os -flto -fwhole-program realmain.o main.a
	main.a(main.o): In function `main':
	main.c:(.text+0x8): undefined reference to `realmain'
	collect2: error: ld returned 1 exit status
Comment 1 Richard Biener 2013-03-05 12:38:51 UTC
    + /usr/bin/avr-gcc -mmcu=atmega328p -Os -flto -fwhole-program realmain.o
main.a

please provide the console output produced when adding -v to the above
commandline.  I suspect that you fall foul of using -fwhole-program without
making the whole program visible to gcc via -flto by means of not using
a compiler that can handle LTO of static archives.

In general, don't use -fwhole-program with -flto, -flto is smart enough
to discover things more intelligently.
Comment 2 Matthijs Kooijman 2013-03-05 12:55:47 UTC
+ /usr/bin/avr-gcc -v -mmcu=atmega328p -Os -flto -fwhole-program realmain.o main.a
Using built-in specs.
COLLECT_GCC=/usr/bin/avr-gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/avr/4.7.2/lto-wrapper
Target: avr
Configured with: ../src/configure -v --enable-languages=c,c++ --prefix=/usr/lib --infodir=/usr/share/info --mandir=/usr/share/man --bindir=/usr/bin --libexecdir=/usr/lib --libdir=/usr/lib --enable-shared --with-system-zlib --enable-long-long --enable-nls --without-included-gettext --disable-libssp --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=avr
Thread model: single
gcc version 4.7.2 (GCC) 
COMPILER_PATH=/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/:/usr/lib/gcc/avr/4.7.2/../../../avr/bin/
LIBRARY_PATH=/usr/lib/gcc/avr/4.7.2/avr5/:/usr/lib/gcc/avr/4.7.2/../../../avr/lib/avr5/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/4.7.2/../../../avr/lib/
COLLECT_GCC_OPTIONS='-v' '-mmcu=atmega328p' '-Os' '-flto' '-fwhole-program'
 /usr/lib/gcc/avr/4.7.2/collect2 -flto -m avr5 -Tdata 0x800100 /usr/lib/gcc/avr/4.7.2/../../../avr/lib/avr5/crtm328p.o -L/usr/lib/gcc/avr/4.7.2/avr5 -L/usr/lib/gcc/avr/4.7.2/../../../avr/lib/avr5 -L/usr/lib/gcc/avr/4.7.2 -L/usr/lib/gcc/avr/4.7.2/../../../avr/lib realmain.o main.a -lgcc -lc -lgcc
 /usr/bin/avr-gcc @/tmp/ccYrSTvi.args
Using built-in specs.
COLLECT_GCC=/usr/bin/avr-gcc
Target: avr
Configured with: ../src/configure -v --enable-languages=c,c++ --prefix=/usr/lib --infodir=/usr/share/info --mandir=/usr/share/man --bindir=/usr/bin --libexecdir=/usr/lib --libdir=/usr/lib --enable-shared --with-system-zlib --enable-long-long --enable-nls --without-included-gettext --disable-libssp --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=avr
Thread model: single
gcc version 4.7.2 (GCC) 
COLLECT_GCC_OPTIONS='-c' '-v' '-mmcu=atmega328p' '-Os' '-fwhole-program' '-fltrans-output-list=/tmp/ccZEu3t3.ltrans.out' '-fwpa'
 /usr/lib/gcc/avr/4.7.2/lto1 -quiet -dumpbase realmain.o -mmcu=atmega328p -auxbase realmain -Os -version -fwhole-program -fltrans-output-list=/tmp/ccZEu3t3.ltrans.out -fwpa @/tmp/ccOsOe32
GNU GIMPLE (GCC) version 4.7.2 (avr)
        compiled by GNU C version 4.7.2, GMP version 5.0.5, MPFR version 3.1.0-p10, MPC version 0.9
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
GNU GIMPLE (GCC) version 4.7.2 (avr)
        compiled by GNU C version 4.7.2, GMP version 5.0.5, MPFR version 3.1.0-p10, MPC version 0.9
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
COMPILER_PATH=/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/:/usr/lib/gcc/avr/4.7.2/../../../avr/bin/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/:/usr/lib/gcc/avr/4.7.2/../../../avr/bin/
LIBRARY_PATH=/usr/lib/gcc/avr/4.7.2/avr5/:/usr/lib/gcc/avr/4.7.2/../../../avr/lib/avr5/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/4.7.2/../../../avr/lib/
COLLECT_GCC_OPTIONS='-c' '-v' '-mmcu=atmega328p' '-Os' '-fwhole-program' '-fltrans-output-list=/tmp/ccZEu3t3.ltrans.out' '-fwpa'
 /usr/bin/avr-gcc @/tmp/ccoysJBM.args
Using built-in specs.
COLLECT_GCC=/usr/bin/avr-gcc
Target: avr
Configured with: ../src/configure -v --enable-languages=c,c++ --prefix=/usr/lib --infodir=/usr/share/info --mandir=/usr/share/man --bindir=/usr/bin --libexecdir=/usr/lib --libdir=/usr/lib --enable-shared --with-system-zlib --enable-long-long --enable-nls --without-included-gettext --disable-libssp --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=avr
Thread model: single
gcc version 4.7.2 (GCC) 
COLLECT_GCC_OPTIONS='-c' '-v' '-mmcu=atmega328p' '-Os' '-fwhole-program' '-fltrans-output-list=/tmp/ccZEu3t3.ltrans.out' '-fltrans' '-o' '/tmp/ccZEu3t3.ltrans0.ltrans.o'
 /usr/lib/gcc/avr/4.7.2/lto1 -quiet -dumpbase ccZEu3t3.ltrans0.o -mmcu=atmega328p -auxbase-strip /tmp/ccZEu3t3.ltrans0.ltrans.o -Os -version -fwhole-program -fltrans-output-list=/tmp/ccZEu3t3.ltrans.out -fltrans @/tmp/ccAudUT3 -o /tmp/ccyDScYi.s
GNU GIMPLE (GCC) version 4.7.2 (avr)
        compiled by GNU C version 4.7.2, GMP version 5.0.5, MPFR version 3.1.0-p10, MPC version 0.9
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
GNU GIMPLE (GCC) version 4.7.2 (avr)
        compiled by GNU C version 4.7.2, GMP version 5.0.5, MPFR version 3.1.0-p10, MPC version 0.9
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
COLLECT_GCC_OPTIONS='-c' '-v' '-mmcu=atmega328p' '-Os' '-fwhole-program' '-fltrans-output-list=/tmp/ccZEu3t3.ltrans.out' '-fltrans' '-o' '/tmp/ccZEu3t3.ltrans0.ltrans.o'
 /usr/lib/gcc/avr/4.7.2/../../../avr/bin/as -mmcu=atmega328p -mno-skip-bug -o /tmp/ccZEu3t3.ltrans0.ltrans.o /tmp/ccyDScYi.s
COMPILER_PATH=/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/:/usr/lib/gcc/avr/4.7.2/../../../avr/bin/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/:/usr/lib/gcc/avr/4.7.2/../../../avr/bin/
LIBRARY_PATH=/usr/lib/gcc/avr/4.7.2/avr5/:/usr/lib/gcc/avr/4.7.2/../../../avr/lib/avr5/:/usr/lib/gcc/avr/4.7.2/:/usr/lib/gcc/avr/4.7.2/../../../avr/lib/
COLLECT_GCC_OPTIONS='-c' '-v' '-mmcu=atmega328p' '-Os' '-fwhole-program' '-fltrans-output-list=/tmp/ccZEu3t3.ltrans.out' '-fltrans' '-o' '/tmp/ccZEu3t3.ltrans0.ltrans.o'
main.a(main.o): In function `main':
main.c:(.text+0x8): undefined reference to `realmain'
collect2: error: ld returned 1 exit status
Comment 3 Matthijs Kooijman 2013-03-05 13:06:18 UTC
Seems I made a wrong observation in my original report: When I link main.o instead of main.a, the problem does _not_ go away. In fact, I can remove a few more flags then, while still keeping the problem around:

$ ./do
	+ rm -f main.a main.o realmain.o
	+ /usr/bin/avr-gcc -c main.c -o main.o
	+ /usr/bin/avr-gcc -c -flto realmain.c -o realmain.o
	+ /usr/bin/avr-gcc -flto -fwhole-program realmain.o main.o
	main.o: In function `main':
	main.c:(.text+0x8): undefined reference to `realmain'
	collect2: error: ld returned 1 exit status

main.c and realmain.c are the same as before.

However, adding -flto to the main.c compilation makes the problem disappear again. I suspect that this means that without -flto, main.o is passed straight to the linker and with -flto it is included in link-time optimization, which would mean your previous analysis still holds.

$ ./do
	+ rm -f *.a main.o realmain.o
	+ /usr/bin/avr-gcc -c -flto main.c -o main.o
	+ /usr/bin/avr-gcc -c -flto realmain.c -o realmain.o
	+ /usr/bin/avr-gcc -flto -fwhole-program realmain.o main.o
Comment 4 Richard Biener 2013-03-05 13:41:01 UTC
Yes.  You are not using a linker-plugin which would end up basically
ignoring -fwhole-program.  With -fwhole-program you are basically lieing to GCC.
Comment 5 Matthijs Kooijman 2013-03-05 14:38:36 UTC
Just for future reference, the problem here seems to be that I'm using -fwhole-program, but the GCC LTO cannot actually look at the whole program. In particular, .a archives and .o object files that were compiled without -flto, are passed directly to the linker and not included in LTO. Since -fwhole-program makes the compiler assume that all files that are included in LTO compose the whole program, the compiler removes symbols that look unused, but then turn up missing in the final link.

So, I shouldn't have been using -fwhole-program, or I should be aware of the above and set externally_visible attributes as needed if I insist on using -fwhole-program.

Ideally, the compiler would ask the linker about which symbols are used in these "non-LTO" objects, which is done by -fuse-linker-plugin (which is implied by -flto). However, on the AVR target, it seems there is no linker plugin (at least not in this particular case), which means that without -fwhole-program, the compiler cannot optimize as much (since it has to assume that all symbols are externally visible).
Comment 6 Matthijs Kooijman 2013-03-05 14:40:15 UTC
w00ps, didn't mean to change the resolution.