Bug 83967 - LTO removes C functions declared as weak in assembler(depending on files order in linking)
Summary: LTO removes C functions declared as weak in assembler(depending on files orde...
Status: RESOLVED MOVED
Alias: None
Product: gcc
Classification: Unclassified
Component: lto (show other bugs)
Version: 7.2.1
: P3 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords: lto, wrong-code
Depends on:
Blocks:
 
Reported: 2018-01-22 10:53 UTC by Rafał Mszal
Modified: 2018-01-26 19:22 UTC (History)
4 users (show)

See Also:
Host: x86_64-linux
Target: arm
Build:
Known to work:
Known to fail:
Last reconfirmed: 2018-01-22 00:00:00


Attachments
Basic example of main funtion with LTO failure. (126 bytes, text/plain)
2018-01-23 08:27 UTC, Rafał Mszal
Details
Makefile that causes failure. (1.38 KB, text/plain)
2018-01-23 08:28 UTC, Rafał Mszal
Details
Startup file for the nRF52 microcontroller. (2.33 KB, text/x-csrc)
2018-01-23 08:31 UTC, Rafał Mszal
Details
self contained demo (97.99 KB, application/gzip)
2018-01-23 15:14 UTC, Bernd K
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Rafał Mszal 2018-01-22 10:53:35 UTC
I've observed strange behaviour of the link-time optimization on GCC 7.2.1, exact version: gcc version 7.2.1 20170904 (release) [ARM/embedded-7-branch revision 255204] (GNU Tools for Arm Embedded Processors 7-2017-q4-major)

I have project that is compiled for the microcontroller. In startup file, that is written in assembler, there are weak definitions of the interrupt handlers. Some of those weak functions are defined in the project C files.

Strange behaviour occurs when I set -flto flag on GCC 7.2.1: during optimization, the defined in C files interrupt functions are perceived as unused and removed(replaced by its weak definitions) - I've checked that in the .map file.

This issue does not occur when startup file is first on the source files list, which is given as an argument to the linker. If I change order of source files, all interrupt handlers from C files, that are placed on the source files list before startup are removed and replaced by weak definitions from startup.

For me it seems that linker is removing interrupt handler functions before noticing its usage in startup file. Or maybe weak function definitions from startup override definitions from C files.

I've checked if this issue happens on the GCC 4.9.3 - it worked fine, so this is rather new behaviour.

I've also confirmed this bug with other developer from Germany: https://www.mikrocontroller.net/topic/443262 
Take a look at the latest posts, which are written in English.

Is this a bug or should I place some extra pragmas/attributes to extort proper linker behaviour?

Please let me know if you want me to prepare some basic examples of this issue.
Comment 1 Richard Biener 2018-01-22 11:52:43 UTC
If you have two weak definitions it is undefined which one takes precedence.

If the C project variant (that gets removed?) is not weak the linker should have
made it used.  But this is only going to work if you build with a linker plugin - so can you double check that (for example pass -fuse-linker-plugin to the link step?).

In any case we'd need something like a testcase to confirm/investigate.

Please also specify the linker you are using.

I assume you are cross compiling from x86_64-linux?
Comment 2 Andrew Pinski 2018-01-22 15:36:00 UTC
Also I was going to say the c function maybe should be marked as used as lto likes to remove unused functions in general.
Comment 3 rguenther@suse.de 2018-01-22 20:47:22 UTC
On January 22, 2018 4:36:00 PM GMT+01:00, "pinskia at gcc dot gnu.org" <gcc-bugzilla@gcc.gnu.org> wrote:
>https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83967
>
>--- Comment #2 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
>Also I was going to say the c function maybe should be marked as used
>as lto
>likes to remove unused functions in general.

If you don't use the linker plugin this is indeed required. But with the linker plugin it shouldn't.
Comment 4 Rafał Mszal 2018-01-23 08:27:42 UTC
Created attachment 43210 [details]
Basic example of main funtion with LTO failure.

This is simple example of observed LTO issue. RTC1_IRQHandler is removed despite it is used to change volatile variable. RTC1_IRQHandler is defined weak in the assembler startup file.
Comment 5 Rafał Mszal 2018-01-23 08:28:52 UTC
Created attachment 43211 [details]
Makefile that causes failure.
Comment 6 Rafał Mszal 2018-01-23 08:31:30 UTC
Created attachment 43212 [details]
Startup file for the nRF52 microcontroller.

This is typical startup for nRF5x microcontrollers. The main thing that is important is the interrupts vector table. All interrupts used in table are declared as weak at the end of file.
Comment 7 Rafał Mszal 2018-01-23 08:38:38 UTC
Thanks for response.

I've send some basic example of the failure. However, it needs some others file to compile and link, like linker, SystemInit function decalration and so on.

I've just send you most important files, now I am working on removing sources that are not important, but this will take me a while, since i need to remove lot of dependencies between those files, makefile and linker.

I've checked 'used' attribute, it doesn't solve the problem. Same with the suggested '-fuse-linker-plugin' flag.
Comment 8 Richard Biener 2018-01-23 09:05:02 UTC
(In reply to Rafał Mszal from comment #4)
> Created attachment 43210 [details]
> Basic example of main funtion with LTO failure.
> 
> This is simple example of observed LTO issue. RTC1_IRQHandler is removed
> despite it is used to change volatile variable. RTC1_IRQHandler is defined
> weak in the assembler startup file.

I don't see it being defined in the startup file.  But I take it should be similar
to

    .weak   NMI_Handler
    .type   NMI_Handler, %function
NMI_Handler:
    b       .
    .size   NMI_Handler, . - NMI_Handler

and the only use is in the startup file as well:

    .section .isr_vector
    .align 2
    .globl __isr_vector
__isr_vector:
    .long   __StackTop                  /* Top of Stack */
    .long   Reset_Handler
    .long   NMI_Handler

and you'd define it in a C file as non-weak (what it does doesn't matter)

void RTC1_IRQHandler(void)
{
}

So a testcase is:

t1.c:
-----

int i;
void Handler()
{
  i = 1;
}
void *Dispatch;
int main(){ ((void (*)(void))Dispatch)(); return i;}

t2.s:
-----

        .file   "t2.s"
        .text
        .weak Handler
        .type   Handler, @function
Handler:
.LFB0:
        .cfi_startproc
        pushq   %rbp
        .cfi_def_cfa_offset 16
        .cfi_offset 6, -16
        movq    %rsp, %rbp
        .cfi_def_cfa_register 6
        popq    %rbp
        .cfi_def_cfa 7, 8
        ret
        .cfi_endproc
.LFE0:
        .size   Handler, .-Handler
        .globl  Dispatch
        .data
        .align 8
        .type   Dispatch, @object
        .size   Dispatch, 8
Dispatch:
        .quad   Handler
        .ident  "GCC: (SUSE Linux) 7.2.1 20171020 [gcc-7-branch revision 253932]"
        .section        .note.GNU-stack,"",@progbits

and then I get

> gcc-7 t2.s t1.c -flto
/tmp/ccGhH7Cp.o:(.data+0x0): undefined reference to `Handler'
collect2: error: ld returned 1 exit status
> gcc-7 t1.c t2.s -flto

in the working case:

1
t1.o 4
190 16b2980a71930688 PREEMPTED_REG Handler
198 16b2980a71930688 PREVAILING_DEF_IRONLY i
194 16b2980a71930688 PREVAILING_DEF main
202 16b2980a71930688 RESOLVED_EXEC Dispatch

in the failing case (GNU ld):

1
t1.o 4
190 801577c96f9ccef3 PREVAILING_DEF_IRONLY Handler
198 801577c96f9ccef3 PREVAILING_DEF_IRONLY i
194 801577c96f9ccef3 PREVAILING_DEF main
202 801577c96f9ccef3 RESOLVED_EXEC Dispatch

which works with gold:

1
t1.o 4
190 680bbd652c7cfe04 PREVAILING_DEF Handler
198 680bbd652c7cfe04 PREVAILING_DEF_IRONLY i
194 680bbd652c7cfe04 PREVAILING_DEF main
202 680bbd652c7cfe04 RESOLVED_EXEC Dispatch

HJ?  This is with binutils 2.29.1, thus confirmed but as a GNU ld issue.
Comment 9 Richard Biener 2018-01-23 09:10:01 UTC
Still not 100% sure which Handler should prevail in resolving the symbol for Dispatch.  Currently it seems the non-weak one prevails which looks correct.

I think in both cases the resolution from GNU ld is bogus.  Take the working one:

190 16b2980a71930688 PREEMPTED_REG Handler

that's clearly wrong -- Handler prevails!

190 801577c96f9ccef3 PREVAILING_DEF_IRONLY Handler

that's wrong because in the assembler file Handler is used to resolve the
symbol at Dispatch.

gold seems to get it correct as PREVAILING_DEF (but not _IRONLY).
Comment 10 rguenther@suse.de 2018-01-23 09:12:05 UTC
On Tue, 23 Jan 2018, hurwic8 at gmail dot com wrote:

> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83967
> 
> --- Comment #7 from Rafał Mszal <hurwic8 at gmail dot com> ---
> Thanks for response.
> 
> I've send some basic example of the failure. However, it needs some others file
> to compile and link, like linker, SystemInit function decalration and so on.
> 
> I've just send you most important files, now I am working on removing sources
> that are not important, but this will take me a while, since i need to remove
> lot of dependencies between those files, makefile and linker.
> 
> I've checked 'used' attribute, it doesn't solve the problem. Same with the
> suggested '-fuse-linker-plugin' flag.

From my analysis the issue is with GNU ld and you could avoid hitting
it by using gold (you can say -fuse-ld=gold to GCC if you have gold
installed)
Comment 11 Bernd K 2018-01-23 15:14:44 UTC
Created attachment 43221 [details]
self contained demo

I have made a minimal self contained project that demonstrates the bug
with arm-none-eabi-gcc compiling for a Cortex M0+ target (Kinetis MKL05), 
it is self contained and does not require anything but the installed 
toolchain itself.

Extract the attached archive KL05_demo_gcc_weak_strong_bug.tar.gz. You 
will find a minimal makefile project, versioned in a git repository with 
only two commits on two branches, one shows the bug and the other 
doesn't.

$ git checkout bad
$ make clean
$ make all

look at the generated .lst file in the build folder and note that the 
Systick handler looks like this, it is the weak default implementation 
from the startup assembly:

000004f4 <SysTick_Handler>:
 4f4:  e7fe        b.n  4f4 <SysTick_Handler>


now checkout the good version

$ git checkout good
$ make clean
$ make all

The systick handler will look like this which is the strong C 
implementation:

0000044c <SysTick_Handler>:
 44c:  4a02        ldr  r2, [pc, #8]  ; (458 <SysTick_Handler+0xc>)
 44e:  6813        ldr  r3, [r2, #0]
 450:  3301        adds  r3, #1
 452:  6013        str  r3, [r2, #0]
 454:  4770        bx  lr
 456:  46c0        nop      ; (mov r8, r8)
 458:  1ffffc00   .word  0x1ffffc00


and the only difference between these two sources is one line in the 
makefile moved to a different position:

$ git diff bad good
diff --git a/Makefile b/Makefile
index 35ff9c6..82e6d47 100644
--- a/Makefile
+++ b/Makefile
@@ -4,9 +4,9 @@
 
 NAME            = example
 
-SRCS           += $(wildcard *.c)
 SRCS           += $(wildcard cmsis/MKL05Z4/*.s)
 SRCS           += $(wildcard cmsis/MKL05Z4/*.c)
+SRCS           += $(wildcard *.c)
 
 INCDIRS                 = cmsis/
 INCDIRS                += cmsis/MKL05Z4/
Comment 12 H.J. Lu 2018-01-26 18:51:18 UTC
(In reply to Richard Biener from comment #8)

> and then I get
> 
> > gcc-7 t2.s t1.c -flto
> /tmp/ccGhH7Cp.o:(.data+0x0): undefined reference to `Handler'
> collect2: error: ld returned 1 exit status
> > gcc-7 t1.c t2.s -flto
> 
> in the working case:
> 
> 1
> t1.o 4
> 190 16b2980a71930688 PREEMPTED_REG Handler
> 198 16b2980a71930688 PREVAILING_DEF_IRONLY i
> 194 16b2980a71930688 PREVAILING_DEF main
> 202 16b2980a71930688 RESOLVED_EXEC Dispatch
> 
> in the failing case (GNU ld):
> 
> 1
> t1.o 4
> 190 801577c96f9ccef3 PREVAILING_DEF_IRONLY Handler
> 198 801577c96f9ccef3 PREVAILING_DEF_IRONLY i
> 194 801577c96f9ccef3 PREVAILING_DEF main
> 202 801577c96f9ccef3 RESOLVED_EXEC Dispatch
> 
> which works with gold:
> 
> 1
> t1.o 4
> 190 680bbd652c7cfe04 PREVAILING_DEF Handler
> 198 680bbd652c7cfe04 PREVAILING_DEF_IRONLY i
> 194 680bbd652c7cfe04 PREVAILING_DEF main
> 202 680bbd652c7cfe04 RESOLVED_EXEC Dispatch
> 
> HJ?  This is with binutils 2.29.1, thus confirmed but as a GNU ld issue.

Yes, I can reproduce it with binutils 2.29.1.  But it was fixed in
2.30.
Comment 13 H.J. Lu 2018-01-26 19:22:29 UTC
(In reply to H.J. Lu from comment #12)
> 
> Yes, I can reproduce it with binutils 2.29.1.  But it was fixed in
> 2.30.

This is:

https://sourceware.org/bugzilla/show_bug.cgi?id=22502