Bug 88750 - [9 Regression] runtime error in statically linked binaries
Summary: [9 Regression] runtime error in statically linked binaries
Status: RESOLVED WORKSFORME
Alias: None
Product: gcc
Classification: Unclassified
Component: fortran (show other bugs)
Version: 9.0
: P3 normal
Target Milestone: 9.0
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-01-07 23:00 UTC by Jürgen Reuter
Modified: 2019-03-17 15:31 UTC (History)
2 users (show)

See Also:
Host:
Target: x86_64-apple-darwin
Build:
Known to work:
Known to fail:
Last reconfirmed: 2019-01-07 00:00:00


Attachments
DYLD_PRINT output non-working example (50.21 KB, text/plain)
2019-01-09 10:50 UTC, Jürgen Reuter
Details
DYLD_PRINT output working example (16.67 KB, text/plain)
2019-01-09 10:51 UTC, Jürgen Reuter
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jürgen Reuter 2019-01-07 23:00:05 UTC
This is really tricky, in our test-suite, we do have a static test. You can get our code from 
http://whizard.tp.nt.uni-siegen.de/tarballs/whizard-nightly-latest.tar.gz
You need gfortran [gcc/g++ for interfaces] as well as the OCaml compiler.
Then you do ./configure, make -j4, make -j4 check. The check you can also just perform `make check` in the build/tests/functional_tests directory. There are two tests, static_1.run and static_2.run which you can separately run as `make check TESTS="static_1.run static_2.run". Both segfault now as 
static_2.exe(65376,0x1186ee5c0) malloc: *** error for object 0x115f18ee0: pointer being freed was not allocated
static_2.exe(65376,0x1186ee5c0) malloc: *** set a breakpoint in malloc_error_break to debug
./static_2.run: line 25: 65376 Abort trap: 6           ./$exe $exe.sin --logfile $exe.log --no-library --no-logging --no-model --no-banner --rebuild
They have been linked using gfortran. The operating system is MAC OS X 10.14.2 and XCode 10.1. I couldn't yet reproduce this on Linux, and I haven't yet been able to boil this down to a simpler example. Unfortunately, on MAC OS X I somehow cannot even run gdb properly. I hope you can help.
Comment 1 Dominique d'Humieres 2019-01-07 23:50:29 UTC
> ... Both segfault now as ...

What is the "now"? Did the tests worked after your update to MAC OS X 10.14.2 or to your latest Xcode?
Comment 2 Jürgen Reuter 2019-01-07 23:53:40 UTC
The tests worked till yesterday, but only today major parts of gcc recompiled because all files in gcc got a new timstamp. I am now trying to recompile also gmp, mpfr and mpc with the llvm-clang from XCode 10.1 (which is from November apparently). But my update to Mojave was before the compilation of gmp, mpfr, and mpc (which happened end of September).
Comment 3 Jürgen Reuter 2019-01-07 23:54:27 UTC
And I am seeing the same problem on another Macbook (Macbook Air) since roughly Christmas (maybe a bit before).
Comment 4 Jürgen Reuter 2019-01-08 05:39:53 UTC
The linker before throws this warning:
ld: warning: direct access in function 'operator new[](unsigned long, std::nothrow_t const&) [clone .cold]' from file '/usr/local/lib/libstdc++.a(new_opvnt.o)' to global weak symbol 'operator new[](unsigned long, std::nothrow_t const&)' from file '/usr/local/lib/libstdc++.a(new_opvnt.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
This could very well be a change in MACOSX or the static stdc++ library.
Comment 5 Jürgen Reuter 2019-01-08 12:26:19 UTC
Indeed, things are more complicated. With a slim build of our software I don't get the error, but only if we link the external library LCIO. This can be downloaded from here: https://github.com/iLCSoft/LCIO/archive/v02-12-01.tar.gz
This is built in a build directory with
cmake .. -DCMAKE_INSTALL_PREFIX=<location> -DCMAKE_C_COMPILER=<gcc> -DCMAKE_CXX_COMPILER=<gnu g++>
and then make, make test, make install
When building WHIZARD, during configure include --enable-lcio (if liblcio can be found in directory known to the (DY)LD_LIBRAY_PATH).
Comment 6 Jürgen Reuter 2019-01-08 18:40:38 UTC
In the linking before I do see the following warning:
ld: warning: direct access in function 'operator new[](unsigned long, std::nothrow_t const&) [clone .cold]' from file '/usr/local/lib/libstdc++.a(new_opvnt.o)' to global weak symbol 'operator new[](unsigned long, std::nothrow_t const&)' from file '/usr/local/lib/libstdc++.a(new_opvnt.o)' means the weak symbol cannot be overridden at runtime. This was likely ca
Not totally sure what that means. The problem appears that libtool links the static stdc++ library into my test binary (via --static-libtool-libs), which causes the warning above, and the malloc runtime error. When I link in by hand the dynamic library via -lstdc++ then the warning goes away as well as the runtime error.
Comment 7 Iain Sandoe 2019-01-08 21:07:40 UTC
(In reply to Jürgen Reuter from comment #6)
> In the linking before I do see the following warning:

> ld: warning: direct access in function 'operator new[](unsigned long,
> std::nothrow_t const&) [clone .cold]' from file
> '/usr/local/lib/libstdc++.a(new_opvnt.o)' to global weak symbol 'operator
> new[](unsigned long, std::nothrow_t const&)' from file
> '/usr/local/lib/libstdc++.a(new_opvnt.o)' means the weak symbol cannot be
> overridden at runtime. This was likely ca

This is almost certainly a "red herring".. (of course, "almost certainly" until proven).

> Not totally sure what that means.

GCC can optimise code to split it into parts that are frequently executed (or 'hot') and parts that are not ('cold').

The externally-visible function typically calls the .cold part to do some setup (or it might be an exceptional or abort path).  In order to facilitate debug, the cold part of the function has a label (__xxxxx.cold:) which is visible to the linker.

In fact, one would never call that symbol from outside the TU in which it's created (it's linker-visible, but not a global) .. 

.. but the linker's complaint here is that if one overrides the externally visible weak "new" symbol that wouldn't override the __new.cold one.

However, the only caller of __new.cold is __new, and therefore if you override __new, nothing will be trying to call __new.cold.  So IMO (probably we should file a radar) the linker warning is bogus (or at least overly conservative).

 The problem appears that libtool links the
> static stdc++ library into my test binary (via --static-libtool-libs), which
> causes the warning above, and the malloc runtime error. When I link in by
> hand the dynamic library via -lstdc++ then the warning goes away as well as
> the runtime error.

* It's hard to deduce what's going on/changed without some specific points as Dominque says...

"then"[working]  and "now" [first known not-working] could do with some definition - plus if possible where the actual failure is (with a backtrace)

* It's generally not possible for folks here to reproduce problems by trying to replicate your builds - this is a volunteer workforce, with highly constrained time resources ;) .. we will need you to narrow things down

* It's quite possible that when you link statically - you are getting the compiler's libstdc++ .. and when you link the shared version, it's being resolved to the one installed in /usr/lib.

Please post as minimum the output of "otool -Lv <binary>" for both the working (lstdc++) and non-working (--static-libtool-libs).
Comment 8 Jürgen Reuter 2019-01-08 21:34:56 UTC
Yes I know: 
here is the non-working library resolution:
static_1.exe:
    lib/libcuttools.dylib (compatibility version 0.0.0, current version 0.0.0)
    lib/libopenloops.dylib (compatibility version 0.0.0, current version 0.0.0)
    lib/liboneloop.dylib (compatibility version 0.0.0, current version 0.0.0)
    lib/libolcommon.dylib (compatibility version 0.0.0, current version 0.0.0)
    lib/librambo.dylib (compatibility version 0.0.0, current version 0.0.0)
    /usr/local/lib/liblcio.2.12.dylib (compatibility version 2.12.0, current version 2.12.1)
    /usr/local/lib/libgfortran.5.dylib (compatibility version 6.0.0, current version 6.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.200.5)
    /usr/local/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/local/lib/libquadmath.0.dylib (compatibility version 1.0.0, current version 1.0.0)

and this is the working one:

static_1.exe:
    lib/libcuttools.dylib (compatibility version 0.0.0, current version 0.0.0)
    lib/libopenloops.dylib (compatibility version 0.0.0, current version 0.0.0)
    lib/liboneloop.dylib (compatibility version 0.0.0, current version 0.0.0)
    lib/libolcommon.dylib (compatibility version 0.0.0, current version 0.0.0)
    lib/librambo.dylib (compatibility version 0.0.0, current version 0.0.0)
    /usr/local/lib/liblcio.2.12.dylib (compatibility version 2.12.0, current version 2.12.1)
    /usr/local/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.26.0)
    /usr/local/lib/libgfortran.5.dylib (compatibility version 6.0.0, current version 6.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.200.5)
    /usr/local/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/local/lib/libquadmath.0.dylib (compatibility version 1.0.0, current version 1.0.0)
Comment 9 Iain Sandoe 2019-01-08 21:50:35 UTC
(In reply to Jürgen Reuter from comment #8)

Thanks!

I've been using gmp-6.1.2, mpfr-3.1.6, mpc-1.1.0 isl-0.20 on all my recent builds (for trunk, gcc-8 and gcc-7)
You don't (I think) mention whether the GCC you're using is trunk or 8 or 7?
I usually bootstrap gap &friends with the compiler, but that ought not to be necessary.

----

I assume that all the code is built with the same compiler?

> here is the non-working library resolution:
> static_1.exe:
>     lib/libcuttools.dylib (compatibility version 0.0.0, current version
> 0.0.0)
>     lib/libopenloops.dylib (compatibility version 0.0.0, current version
> 0.0.0)
>     lib/liboneloop.dylib (compatibility version 0.0.0, current version 0.0.0)
>     lib/libolcommon.dylib (compatibility version 0.0.0, current version
> 0.0.0)
>     lib/librambo.dylib (compatibility version 0.0.0, current version 0.0.0)
>     /usr/local/lib/liblcio.2.12.dylib (compatibility version 2.12.0, current
> version 2.12.1)
>     /usr/local/lib/libgfortran.5.dylib (compatibility version 6.0.0, current
> version 6.0.0)
>     /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version
> 1252.200.5)
>     /usr/local/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current
> version 1.0.0)
>     /usr/local/lib/libquadmath.0.dylib (compatibility version 1.0.0, current
> version 1.0.0)
> 
> and this is the working one:
> 
> static_1.exe:
>     lib/libcuttools.dylib (compatibility version 0.0.0, current version
> 0.0.0)
>     lib/libopenloops.dylib (compatibility version 0.0.0, current version
> 0.0.0)
>     lib/liboneloop.dylib (compatibility version 0.0.0, current version 0.0.0)
>     lib/libolcommon.dylib (compatibility version 0.0.0, current version
> 0.0.0)
>     lib/librambo.dylib (compatibility version 0.0.0, current version 0.0.0)
>     /usr/local/lib/liblcio.2.12.dylib (compatibility version 2.12.0, current
> version 2.12.1)
>     /usr/local/lib/libstdc++.6.dylib (compatibility version 7.0.0, current
> version 7.26.0)

^^^ this is the installation from the same compiler that you used to build the non-working example above?

>     /usr/local/lib/libgfortran.5.dylib (compatibility version 6.0.0, current
> version 6.0.0)
>     /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version
> 1252.200.5)
>     /usr/local/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current
> version 1.0.0)
>     /usr/local/lib/libquadmath.0.dylib (compatibility version 1.0.0, current
> version 1.0.0)

* If you invoke the exe with 
DYLD_PRINT_LIBRARIES=1 <exe>
you can confirm that it is, indeed, using the libraries listed there...

* So the linker/assembler/otool/ar/nm/strip/etc you're using come from Xcode 10.1?

* GCC binaries built with debug enabled are debugable by lldb, if you don't have a working GDB.
(I use -Og -g3 as build options, FWIW)
Comment 10 Jürgen Reuter 2019-01-08 21:52:43 UTC
The trunk, svn 267657, all newest versions of gmp, mpfr, mpc. It seems that the problem is also solved when I use the libtool flag -static instead of -static-libtool-libs for libtool to build these executables.
Comment 11 Iain Sandoe 2019-01-08 22:00:59 UTC
(In reply to Jürgen Reuter from comment #10)
> The trunk, svn 267657, all newest versions of gmp, mpfr, mpc. It seems that
> the problem is also solved when I use the libtool flag -static instead of
> -static-libtool-libs for libtool to build these executables.

Thanks, can you identify a "working" svn #?

Is it possible to attach a transcript of the build(s) with the three cases - two working and one not?

We did make a number of commits over Christmas and, if there's a regression, would like to pin it down as soon as possible.
Comment 12 Jürgen Reuter 2019-01-08 22:03:31 UTC
No, unfortunately a working svn # is difficult, I first observed it by doing svn up on another Macbook around Christmas. What do you mean by transcripts?
Comment 13 Iain Sandoe 2019-01-08 22:06:25 UTC
(In reply to Jürgen Reuter from comment #12)
> No, unfortunately a working svn # is difficult, I first observed it by doing
> svn up on another Macbook around Christmas. 

hmm ... that's tricky - a busy time .. so looks like well have to try to detective work.

> What do you mean by transcripts?

the output of the build process, for a clean build (most interesting will be link lines etc.) ..
Comment 14 Jürgen Reuter 2019-01-08 22:33:47 UTC
Well, it seems that r267488 from Dec 31 was still working, on the other hand, I saw a problem on the other MACbook definitely around at latest Dec 26 or so. Probably before Christmas. It might be that small updates did not trigger a complete recompilation of the trunk? 
This one is failing:
gfortran -g -O2 -Wl,-rpath -Wl,/usr/local/packages/OpenLoops/lib -o static_1.exe .libs/static_1.exe_prclib_dispatcher.o  -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hepmc -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/lcio -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hoppet -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/looptools -L/usr/local/packages/OpenLoops/lib -L/usr/local/lib -L../src ./.libs/static_1_lib.a -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/models /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core/.libs/libwhizard_main.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/.libs/libomega.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/omega/src/.libs/libomega_core.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/.libs/libwhizard.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/vamp/src/.libs/libvamp.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/circe1/src/.libs/libcirce1.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/circe2/src/.libs/libcirce2.a -lcuttools -lopenloops -loneloop -lolcommon -lrambo /usr/local/lib/libLHAPDF.a /usr/local//lib/libHepMC.a -llcio /usr/local/lib/libstdc++.a -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src/.libs -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/libsupc++/.libs -lm 

while that one is working:

gfortran -g -O2 -Wl,-rpath -Wl,/usr/local/packages/OpenLoops/lib -o static_1.exe .libs/static_1.exe_prclib_dispatcher.o  -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hepmc -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/lcio -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hoppet -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/looptools -L/usr/local/packages/OpenLoops/lib -L/usr/local/lib -L../src ./.libs/static_1_lib.a -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/models /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core/.libs/libwhizard_main.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/.libs/libomega.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/omega/src/.libs/libomega_core.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/.libs/libwhizard.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/vamp/src/.libs/libvamp.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/circe1/src/.libs/libcirce1.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/circe2/src/.libs/libcirce2.a -lcuttools -lopenloops -loneloop -lolcommon -lrambo /usr/local/lib/libLHAPDF.a -L/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0 -L/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0/../../.. -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src/.libs -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/libsupc++/.libs -lSystem -lgcc_ext.10.5 /usr/local//lib/libHepMC.a -lstdc++ -llcio -lm

But that's probably not very helpful for you.
Comment 15 Iain Sandoe 2019-01-08 23:08:03 UTC
(In reply to Jürgen Reuter from comment #14)
> Well, it seems that r267488 from Dec 31 was still working, on the other
> hand, I saw a problem on the other MACbook definitely around at latest Dec
> 26 or so. Probably before Christmas. It might be that small updates did not
> trigger a complete recompilation of the trunk? 

Have you done a complete clean rebuild of both compiler and application?
( partial rebuilds are hard to analyse at the best of times ).

31st is after most of the Darwin-specific changes went in.

I will look at the transcripts tomorrow/
Comment 16 Jürgen Reuter 2019-01-09 05:12:13 UTC
Yes, after the problem occurred, I did a completely clean new build of gmp, mpfr, mpc, gcc (configured with ../configure --prefix=/usr/local/ --with-gmp=/usr/local/ --with-mpfr=/usr/local/ --with-mpc=/usr/local/ --enable-checking=release --enable-languages=c,c++,fortran,lto),
all the tools our software depends, and our software. It turns out that external C++ libraries linked into our (Fortran) project via bind(C) are not a problem if they have been built via libtool, such that a .dylib, a .a and a .la file are present. The two projects that have problem either exist as .dylib and .a produced by hand-written configure and makefiles (i.e. not using autotools), or only as dynamic libraries produced via cmake and make.
Comment 17 Iain Sandoe 2019-01-09 08:09:43 UTC
(In reply to Jürgen Reuter from comment #16)
> Yes, after the problem occurred, I did a completely clean new build of gmp,
> mpfr, mpc, gcc (configured with ../configure --prefix=/usr/local/
> --with-gmp=/usr/local/ --with-mpfr=/usr/local/ --with-mpc=/usr/local/
> --enable-checking=release --enable-languages=c,c++,fortran,lto),
> all the tools our software depends, and our software.

OK, FWIW (thinking a bit more last night) if you examine the logs from building GCC, you will see the same linker complaint in the log for building libstdc++.dylib.  Which kinda reinforces the expectation that this is not the source of the problem.  However, I'm thinking to try and construct some small experiment to check that the newer ld64 doesn't do something active as well as complain.

> It turns out that
> external C++ libraries linked into our (Fortran) project via bind(C)

I might be wrong, but suspect there was some change to the C binding around that time too - but I also recall seeing a recent patch go by to fix a problem in that area (but not sure if it's been applied yet).  Will let Dominique comment on that.

> are not
> a problem if they have been built via libtool, such that a .dylib, a .a and
> a .la file are present. The two projects that have problem either exist as
> .dylib and .a produced by hand-written configure and makefiles (i.e. not
> using autotools), or only as dynamic libraries produced via cmake and make.

That's an interesting observation, what we need is to find the specific difference in the output exe.

* Narrowing this down by knowing where and what causes the problem will become important at some point - so a debug build and lldb session could be a useful next step.

* as a general rule, it's also useful to see if an -O0 build exhibits the problem - in case its an optimisation  issue.
Comment 18 Iain Sandoe 2019-01-09 08:51:16 UTC
(In reply to Jürgen Reuter from comment #14)

does the application use exceptions?

> This one is failing:
> gfortran -g -O2 -Wl,-rpath -Wl,/usr/local/packages/OpenLoops/lib -o
> static_1.exe .libs/static_1.exe_prclib_dispatcher.o 

<snip>

> /usr/local/lib/libstdc++.a

^^^ please confirm that this is from the "current compiler build".

<snip>

> -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/
> libsupc++/.libs -lm 

^^^^ note - no "-lSystem -lgcc_ext.10.5" (which is what I'd expect).

> 
> while that one is working:
> 
> gfortran -g -O2 -Wl,-rpath -Wl,/usr/local/packages/OpenLoops/lib -o
> static_1.exe .libs/static_1.exe_prclib_dispatcher.o 

<snip>

> libsupc++/.libs -lSystem -lgcc_ext.10.5 /usr/local//lib/libHepMC.a -lstdc++
> -llcio -lm

^^^^^^^ this looks like the build process in this case is adding libs that the compiler driver normally adds ( they are not present in the case above ).

* If you can extract these two fortran link lines - and then execute them separately in the build dir with "-v" so that we can see the output of the compiler-driver's internal link line and what its search paths are.

* According to your posted otool output, the version of libstdc++.dylib that is bound is the one in /usr/local/lib/ which is where you pick up the static lib in the non-working case.

* The object files used to build the static (.a) and dynamic (.dylib) versions of libstdc++ are the same, so we really need to pin down where the issue occurs.

* DYLD_PRINT_LIBRARIES=1 DYLD_PRINT_BINDINGS=1 <exe> 
will show you which libraries are used, and from which library each symbol is resolved - it probably will produce a lot of output..
Comment 19 Jürgen Reuter 2019-01-09 10:44:14 UTC
(In reply to Iain Sandoe from comment #18)
> (In reply to Jürgen Reuter from comment #14)
> 
> does the application use exceptions?

No exceptions, only a poor man's C signal catcher. 

> 
> > /usr/local/lib/libstdc++.a
> 
> ^^^ please confirm that this is from the "current compiler build".
> 

Yes, they are the same. Unfortunately, there is no uninstall target for gcc, but all stdc++ libraries in /usr/local/lib are from my Jan 8 clean building. 


> 
> ^^^^^^^ this looks like the build process in this case is adding libs that
> the compiler driver normally adds ( they are not present in the case above ).
> 

Yes, that is for a different reason, a different build with a tutorial C and C++ wrapper for our code, but they don't hurt here.


> * If you can extract these two fortran link lines - and then execute them
> separately in the build dir with "-v" so that we can see the output of the
> compiler-driver's internal link line and what its search paths are.

This is the output for the non-working linking:
$ gfortran -g -O2 -Wl,-rpath -Wl,/usr/local/packages/OpenLoops/lib -o static_1.exe .libs/static_1.exe_prclib_dispatcher.o  -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hepmc -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/lcio -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hoppet -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/looptools -L/usr/local/packages/OpenLoops/lib -L/usr/local/lib -L../src ./.libs/static_1_lib.a -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/models /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core/.libs/libwhizard_main.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/.libs/libomega.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/omega/src/.libs/libomega_core.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/.libs/libwhizard.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/vamp/src/.libs/libvamp.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/circe1/src/.libs/libcirce1.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/circe2/src/.libs/libcirce2.a -lcuttools -lopenloops -loneloop -lolcommon -lrambo /usr/local/lib/libLHAPDF.a /usr/local//lib/libHepMC.a -llcio /usr/local/lib/libstdc++.a -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src/.libs -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/libsupc++/.libs -lm -v
Driving: gfortran -g -O2 -Wl,-rpath -Wl,/usr/local/packages/OpenLoops/lib -o static_1.exe .libs/static_1.exe_prclib_dispatcher.o -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hepmc -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/lcio -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hoppet -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/looptools -L/usr/local/packages/OpenLoops/lib -L/usr/local/lib -L../src ./.libs/static_1_lib.a -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/models /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core/.libs/libwhizard_main.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/.libs/libomega.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/omega/src/.libs/libomega_core.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/.libs/libwhizard.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/vamp/src/.libs/libvamp.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/circe1/src/.libs/libcirce1.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/circe2/src/.libs/libcirce2.a -lcuttools -lopenloops -loneloop -lolcommon -lrambo /usr/local/lib/libLHAPDF.a /usr/local//lib/libHepMC.a -llcio /usr/local/lib/libstdc++.a -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src/.libs -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/libsupc++/.libs -lm -v -mmacosx-version-min=10.14.0 -asm_macosx_version_min=10.14 -l gfortran -shared-libgcc
Using built-in specs.
COLLECT_GCC=gfortran
COLLECT_LTO_WRAPPER=/usr/local/libexec/gcc/x86_64-apple-darwin18.2.0/9.0.0/lto-wrapper
Target: x86_64-apple-darwin18.2.0
Configured with: ../configure --prefix=/usr/local/ --with-gmp=/usr/local/ --with-mpfr=/usr/local/ --with-mpc=/usr/local/ --enable-checking=release --enable-languages=c,c++,fortran,lto
Thread model: posix
gcc version 9.0.0 20190107 (experimental) (GCC) 
Reading specs from /usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0/../../../libgfortran.spec
rename spec lib to liborig
COLLECT_GCC_OPTIONS='-g' '-O2' '-o' 'static_1.exe' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hepmc' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/lcio' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hoppet' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/looptools' '-L/usr/local/packages/OpenLoops/lib' '-L/usr/local/lib' '-L../src' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/models' '-L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src' '-L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src/.libs' '-L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/libsupc++/.libs' '-v' '-mmacosx-version-min=10.14.0' '-asm_macosx_version_min=10.14' '-shared-libgcc' '-mtune=core2'
COMPILER_PATH=/usr/local/libexec/gcc/x86_64-apple-darwin18.2.0/9.0.0/:/usr/local/libexec/gcc/x86_64-apple-darwin18.2.0/9.0.0/:/usr/local/libexec/gcc/x86_64-apple-darwin18.2.0/:/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0/:/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/
LIBRARY_PATH=/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0/:/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0/../../../
COLLECT_GCC_OPTIONS='-g' '-O2' '-o' 'static_1.exe' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hepmc' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/lcio' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hoppet' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/looptools' '-L/usr/local/packages/OpenLoops/lib' '-L/usr/local/lib' '-L../src' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/models' '-L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src' '-L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src/.libs' '-L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/libsupc++/.libs' '-v' '-mmacosx-version-min=10.14.0' '-asm_macosx_version_min=10.14' '-shared-libgcc' '-mtune=core2'
 /usr/local/libexec/gcc/x86_64-apple-darwin18.2.0/9.0.0/collect2 -dynamic -arch x86_64 -macosx_version_min 10.14.0 -weak_reference_mismatches non-weak -o static_1.exe -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hepmc -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/lcio -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hoppet -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/looptools -L/usr/local/packages/OpenLoops/lib -L/usr/local/lib -L../src -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/models -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src/.libs -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/libsupc++/.libs -L/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0 -L/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0/../../.. -rpath /usr/local/packages/OpenLoops/lib .libs/static_1.exe_prclib_dispatcher.o ./.libs/static_1_lib.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core/.libs/libwhizard_main.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/.libs/libomega.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/omega/src/.libs/libomega_core.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/.libs/libwhizard.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/vamp/src/.libs/libvamp.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/circe1/src/.libs/libcirce1.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/circe2/src/.libs/libcirce2.a -lcuttools -lopenloops -loneloop -lolcommon -lrambo /usr/local/lib/libLHAPDF.a /usr/local//lib/libHepMC.a -llcio /usr/local/lib/libstdc++.a -lgfortran -no_compact_unwind -lSystem -lgcc_ext.10.5 -lgcc -lquadmath -lm -lgcc_ext.10.5 -lgcc -lSystem -v -idsym
collect2 version 9.0.0 20190107 (experimental)
/usr/bin/ld -dynamic -arch x86_64 -macosx_version_min 10.14.0 -weak_reference_mismatches non-weak -o static_1.exe -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hepmc -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/lcio -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hoppet -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/looptools -L/usr/local/packages/OpenLoops/lib -L/usr/local/lib -L../src -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/models -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src/.libs -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/libsupc++/.libs -L/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0 -L/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0/../../.. -rpath /usr/local/packages/OpenLoops/lib .libs/static_1.exe_prclib_dispatcher.o ./.libs/static_1_lib.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core/.libs/libwhizard_main.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/.libs/libomega.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/omega/src/.libs/libomega_core.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/.libs/libwhizard.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/vamp/src/.libs/libvamp.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/circe1/src/.libs/libcirce1.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/circe2/src/.libs/libcirce2.a -lcuttools -lopenloops -loneloop -lolcommon -lrambo /usr/local/lib/libLHAPDF.a /usr/local//lib/libHepMC.a -llcio /usr/local/lib/libstdc++.a -lgfortran -no_compact_unwind -lSystem -lgcc_ext.10.5 -lgcc -lquadmath -lm -lgcc_ext.10.5 -lgcc -lSystem -v
@(#)PROGRAM:ld  PROJECT:ld64-409.12
BUILD 17:47:51 Sep 25 2018
configured to support archs: armv6 armv7 armv7s arm64 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em arm64e arm64_32
ld: warning: directory not found for option '-L../src'
Library search paths:
	/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core
	/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src
	/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hepmc
	/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/lcio
	/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hoppet
	/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/looptools
	/usr/local/packages/OpenLoops/lib
	/usr/local/lib
	/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/models
	/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src
	/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src/.libs
	/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/libsupc++/.libs
	/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0
	/usr/local/lib
	/usr/lib
	/usr/local/lib
Framework search paths:
	/Library/Frameworks/
	/System/Library/Frameworks/
ld: warning: direct access in function 'operator new[](unsigned long, std::nothrow_t const&) [clone .cold]' from file '/usr/local/lib/libstdc++.a(new_opvnt.o)' to global weak symbol 'operator new[](unsigned long, std::nothrow_t const&)' from file '/usr/local/lib/libstdc++.a(new_opvnt.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
reuter@vpn-005:~/Physik/whizard/trunk/_build_quasi_naked/tests/functional_tests$ 


And this is the one for the working linking:

$ gfortran -g -O2 -Wl,-rpath -Wl,/usr/local/packages/OpenLoops/lib -o static_1.exe .libs/static_1.exe_prclib_dispatcher.o  -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hepmc -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/lcio -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hoppet -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/looptools -L/usr/local/packages/OpenLoops/lib -L/usr/local/lib -L../src ./.libs/static_1_lib.a -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/models /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core/.libs/libwhizard_main.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/.libs/libomega.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/omega/src/.libs/libomega_core.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/.libs/libwhizard.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/vamp/src/.libs/libvamp.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/circe1/src/.libs/libcirce1.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/circe2/src/.libs/libcirce2.a -lcuttools -lopenloops -loneloop -lolcommon -lrambo /usr/local/lib/libLHAPDF.a -L/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0 -L/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0/../../.. -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src/.libs -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/libsupc++/.libs -lSystem -lgcc_ext.10.5 /usr/local//lib/libHepMC.a -llcio -lstdc++ -lm -v
Driving: gfortran -g -O2 -Wl,-rpath -Wl,/usr/local/packages/OpenLoops/lib -o static_1.exe .libs/static_1.exe_prclib_dispatcher.o -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hepmc -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/lcio -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hoppet -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/looptools -L/usr/local/packages/OpenLoops/lib -L/usr/local/lib -L../src ./.libs/static_1_lib.a -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/models /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core/.libs/libwhizard_main.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/.libs/libomega.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/omega/src/.libs/libomega_core.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/.libs/libwhizard.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/vamp/src/.libs/libvamp.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/circe1/src/.libs/libcirce1.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/circe2/src/.libs/libcirce2.a -lcuttools -lopenloops -loneloop -lolcommon -lrambo /usr/local/lib/libLHAPDF.a -L/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0 -L/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0/../../.. -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src/.libs -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/libsupc++/.libs -lSystem -lgcc_ext.10.5 /usr/local//lib/libHepMC.a -llcio -lstdc++ -lm -v -mmacosx-version-min=10.14.0 -asm_macosx_version_min=10.14 -l gfortran -shared-libgcc
Using built-in specs.
COLLECT_GCC=gfortran
COLLECT_LTO_WRAPPER=/usr/local/libexec/gcc/x86_64-apple-darwin18.2.0/9.0.0/lto-wrapper
Target: x86_64-apple-darwin18.2.0
Configured with: ../configure --prefix=/usr/local/ --with-gmp=/usr/local/ --with-mpfr=/usr/local/ --with-mpc=/usr/local/ --enable-checking=release --enable-languages=c,c++,fortran,lto
Thread model: posix
gcc version 9.0.0 20190107 (experimental) (GCC) 
Reading specs from /usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0/../../../libgfortran.spec
rename spec lib to liborig
COLLECT_GCC_OPTIONS='-g' '-O2' '-o' 'static_1.exe' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hepmc' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/lcio' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hoppet' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/looptools' '-L/usr/local/packages/OpenLoops/lib' '-L/usr/local/lib' '-L../src' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/models' '-L/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0' '-L/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0/../../..' '-L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src' '-L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src/.libs' '-L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/libsupc++/.libs' '-v' '-mmacosx-version-min=10.14.0' '-asm_macosx_version_min=10.14' '-shared-libgcc' '-mtune=core2'
COMPILER_PATH=/usr/local/libexec/gcc/x86_64-apple-darwin18.2.0/9.0.0/:/usr/local/libexec/gcc/x86_64-apple-darwin18.2.0/9.0.0/:/usr/local/libexec/gcc/x86_64-apple-darwin18.2.0/:/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0/:/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/
LIBRARY_PATH=/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0/:/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0/../../../
COLLECT_GCC_OPTIONS='-g' '-O2' '-o' 'static_1.exe' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hepmc' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/lcio' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hoppet' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/looptools' '-L/usr/local/packages/OpenLoops/lib' '-L/usr/local/lib' '-L../src' '-L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/models' '-L/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0' '-L/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0/../../..' '-L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src' '-L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src/.libs' '-L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/libsupc++/.libs' '-v' '-mmacosx-version-min=10.14.0' '-asm_macosx_version_min=10.14' '-shared-libgcc' '-mtune=core2'
 /usr/local/libexec/gcc/x86_64-apple-darwin18.2.0/9.0.0/collect2 -dynamic -arch x86_64 -macosx_version_min 10.14.0 -weak_reference_mismatches non-weak -o static_1.exe -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hepmc -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/lcio -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hoppet -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/looptools -L/usr/local/packages/OpenLoops/lib -L/usr/local/lib -L../src -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/models -L/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0 -L/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0/../../.. -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src/.libs -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/libsupc++/.libs -L/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0 -L/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0/../../.. -rpath /usr/local/packages/OpenLoops/lib .libs/static_1.exe_prclib_dispatcher.o ./.libs/static_1_lib.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core/.libs/libwhizard_main.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/.libs/libomega.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/omega/src/.libs/libomega_core.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/.libs/libwhizard.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/vamp/src/.libs/libvamp.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/circe1/src/.libs/libcirce1.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/circe2/src/.libs/libcirce2.a -lcuttools -lopenloops -loneloop -lolcommon -lrambo /usr/local/lib/libLHAPDF.a -lSystem -lgcc_ext.10.5 /usr/local//lib/libHepMC.a -llcio -lstdc++ -lgfortran -no_compact_unwind -lSystem -lgcc_ext.10.5 -lgcc -lquadmath -lm -lgcc_ext.10.5 -lgcc -lSystem -v -idsym
collect2 version 9.0.0 20190107 (experimental)
/usr/bin/ld -dynamic -arch x86_64 -macosx_version_min 10.14.0 -weak_reference_mismatches non-weak -o static_1.exe -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hepmc -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/lcio -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hoppet -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/looptools -L/usr/local/packages/OpenLoops/lib -L/usr/local/lib -L../src -L/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/models -L/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0 -L/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0/../../.. -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src/.libs -L/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/libsupc++/.libs -L/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0 -L/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0/../../.. -rpath /usr/local/packages/OpenLoops/lib .libs/static_1.exe_prclib_dispatcher.o ./.libs/static_1_lib.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core/.libs/libwhizard_main.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/.libs/libomega.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/omega/src/.libs/libomega_core.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/.libs/libwhizard.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/vamp/src/.libs/libvamp.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/circe1/src/.libs/libcirce1.a /Users/reuter/Physik/whizard/trunk/_build_quasi_naked/circe2/src/.libs/libcirce2.a -lcuttools -lopenloops -loneloop -lolcommon -lrambo /usr/local/lib/libLHAPDF.a -lSystem -lgcc_ext.10.5 /usr/local//lib/libHepMC.a -llcio -lstdc++ -lgfortran -no_compact_unwind -lSystem -lgcc_ext.10.5 -lgcc -lquadmath -lm -lgcc_ext.10.5 -lgcc -lSystem -v
@(#)PROGRAM:ld  PROJECT:ld64-409.12
BUILD 17:47:51 Sep 25 2018
configured to support archs: armv6 armv7 armv7s arm64 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em arm64e arm64_32
ld: warning: directory not found for option '-L../src'
Library search paths:
	/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/whizard-core
	/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src
	/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hepmc
	/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/lcio
	/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/hoppet
	/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/looptools
	/usr/local/packages/OpenLoops/lib
	/usr/local/lib
	/Users/reuter/Physik/whizard/trunk/_build_quasi_naked/src/models
	/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0
	/usr/local/lib
	/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src
	/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/src/.libs
	/usr/local/packages/gcc_9.0/_build/x86_64-apple-darwin18.2.0/libstdc++-v3/libsupc++/.libs
	/usr/local/lib/gcc/x86_64-apple-darwin18.2.0/9.0.0
	/usr/local/lib
	/usr/lib
	/usr/local/lib
Framework search paths:
	/Library/Frameworks/
	/System/Library/Frameworks/




> * According to your posted otool output, the version of libstdc++.dylib that
> is bound is the one in /usr/local/lib/ which is where you pick up the static
> lib in the non-working case.
> 
> * The object files used to build the static (.a) and dynamic (.dylib)
> versions of libstdc++ are the same, so we really need to pin down where the
> issue occurs.
> 
> * DYLD_PRINT_LIBRARIES=1 DYLD_PRINT_BINDINGS=1 <exe> 
> will show you which libraries are used, and from which library each symbol
> is resolved - it probably will produce a lot of output..

I will attach this output.
Comment 20 Jürgen Reuter 2019-01-09 10:50:58 UTC
Created attachment 45387 [details]
DYLD_PRINT output non-working example

DYLD_PRINT_LIBRARIES=1 DYLD_PRINT_BINDINGS=1 ./static_1.exe > non_working_output 2>&1
Comment 21 Jürgen Reuter 2019-01-09 10:51:53 UTC
Created attachment 45388 [details]
DYLD_PRINT output working example

DYLD_PRINT_LIBRARIES=1 DYLD_PRINT_BINDINGS=1 ./static_1.exe > working_output 2>&1
Comment 22 Jürgen Reuter 2019-01-09 11:01:45 UTC
This is the output from the lldb command (but this was not a debug build of gcc yet):
$ lldb ./static_1.exe
(lldb) target create "./static_1.exe"
Current executable set to './static_1.exe' (x86_64).
(lldb) run
Process 36799 launched: './static_1.exe' (x86_64)
static_1.exe(36799,0x1048f75c0) malloc: *** error for object 0x105c5eee0: pointer being freed was not allocated
static_1.exe(36799,0x1048f75c0) malloc: *** set a breakpoint in malloc_error_break to debug
Process 36799 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
    frame #0: 0x00007fff5a2d023e libsystem_kernel.dylib`__pthread_kill + 10
libsystem_kernel.dylib`__pthread_kill:
->  0x7fff5a2d023e <+10>: jae    0x7fff5a2d0248            ; <+20>
    0x7fff5a2d0240 <+12>: movq   %rax, %rdi
    0x7fff5a2d0243 <+15>: jmp    0x7fff5a2ca3b7            ; cerror_nocancel
    0x7fff5a2d0248 <+20>: retq   
Target 0: (static_1.exe) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
  * frame #0: 0x00007fff5a2d023e libsystem_kernel.dylib`__pthread_kill + 10
    frame #1: 0x00007fff5a386c1c libsystem_pthread.dylib`pthread_kill + 285
    frame #2: 0x00007fff5a2391c9 libsystem_c.dylib`abort + 127
    frame #3: 0x00007fff5a3486e2 libsystem_malloc.dylib`malloc_vreport + 545
    frame #4: 0x00007fff5a3484a3 libsystem_malloc.dylib`malloc_report + 152
    frame #5: 0x0000000100929c84 static_1.exe`std::locale::_Impl::~_Impl(this=0x0000000105c5f0a0) at locale.cc:243
    frame #6: 0x0000000100929d8e static_1.exe`std::locale::operator=(this=0x0000000105c611c0, __other=0x00007ffeefbfdad8) at locale_classes.h:568
    frame #7: 0x0000000100927aec static_1.exe`std::ios_base::_M_init(this=0x0000000105c610f0) at ios_locale.cc:44
    frame #8: 0x000000010096cef1 static_1.exe`std::basic_ios<char, std::char_traits<char> >::init(this=0x0000000105c610f0, __sb=0x0000000105c60840) at basic_ios.tcc:129
    frame #9: 0x0000000105afcdf9 libstdc++.6.dylib`std::ios_base::Init::Init() + 681
    frame #10: 0x0000000105ad30a0 libsio.2.12.dylib`_GLOBAL__sub_I_SIO_blockManager.cc + 16
    frame #11: 0x0000000104859cc8 dyld`ImageLoaderMachO::doModInitFunctions(ImageLoader::LinkContext const&) + 518
    frame #12: 0x0000000104859ec6 dyld`ImageLoaderMachO::doInitialization(ImageLoader::LinkContext const&) + 40
    frame #13: 0x00000001048550da dyld`ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 358
    frame #14: 0x000000010485506d dyld`ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 249
    frame #15: 0x000000010485506d dyld`ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, char const*, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 249
    frame #16: 0x0000000104854254 dyld`ImageLoader::processInitializers(ImageLoader::LinkContext const&, unsigned int, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 134
    frame #17: 0x00000001048542e8 dyld`ImageLoader::runInitializers(ImageLoader::LinkContext const&, ImageLoader::InitializerTimingList&) + 74
    frame #18: 0x0000000104843774 dyld`dyld::initializeMainExecutable() + 199
    frame #19: 0x000000010484878f dyld`dyld::_main(macho_header const*, unsigned long, int, char const**, char const**, char const**, unsigned long*) + 6237
    frame #20: 0x00000001048424f6 dyld`dyldbootstrap::start(macho_header const*, int, char const**, long, macho_header const*, unsigned long*) + 1154
    frame #21: 0x0000000104842036 dyld`_dyld_start + 54
Comment 23 Iain Sandoe 2019-01-09 11:16:26 UTC
(In reply to Jürgen Reuter from comment #22)
> This is the output from the lldb command (but this was not a debug build of
> gcc yet):
> $ lldb ./static_1.exe
> (lldb) target create "./static_1.exe"
> Current executable set to './static_1.exe' (x86_64).
> (lldb) run

<snip>

> __sb=0x0000000105c60840) at basic_ios.tcc:129
>     frame #9: 0x0000000105afcdf9
> libstdc++.6.dylib`std::ios_base::Init::Init() + 681
>     frame #10: 0x0000000105ad30a0

^^^^ so, you have a combination of things linking libstdc++ statically and dynamically .. that seems fragile at best.

Having said that - the tricky thing now is to determine what has "broken" (it's probably going to be hard without a "before" and "after" case).
Comment 24 Richard Biener 2019-01-09 11:23:26 UTC
(In reply to Iain Sandoe from comment #23)
> (In reply to Jürgen Reuter from comment #22)
> > This is the output from the lldb command (but this was not a debug build of
> > gcc yet):
> > $ lldb ./static_1.exe
> > (lldb) target create "./static_1.exe"
> > Current executable set to './static_1.exe' (x86_64).
> > (lldb) run
> 
> <snip>
> 
> > __sb=0x0000000105c60840) at basic_ios.tcc:129
> >     frame #9: 0x0000000105afcdf9
> > libstdc++.6.dylib`std::ios_base::Init::Init() + 681
> >     frame #10: 0x0000000105ad30a0
> 
> ^^^^ so, you have a combination of things linking libstdc++ statically and
> dynamically .. that seems fragile at best.
> 
> Having said that - the tricky thing now is to determine what has "broken"
> (it's probably going to be hard without a "before" and "after" case).

Indeed - somehow you didn't get a statically linked executable.  Quoting the
full final link command would be interesting.
Comment 25 Jürgen Reuter 2019-01-09 11:35:28 UTC
(In reply to Richard Biener from comment #24)
> (In reply to Iain Sandoe from comment #23)
> > (In reply to Jürgen Reuter from comment #22)

> 
> Indeed - somehow you didn't get a statically linked executable.  Quoting the
> full final link command would be interesting.

The full link commands can be found here, I believe: 
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88750#c14

Our code generates code for particle physics simulations in the form of dynamic libraries that get linked and loaded. For batch clusters, we attempted to provide static binaries for these simulations, however, we have order 10-15 external libraries that can be linked to our code (which are partially mandatory). There are some of them which only exist as dynamic libraries, so there our approach cannot result in a purely static binary. The static stdc++ library is sucked in via the libtool link mode/flag -static-libtool-libs while the dynamic ones are sucked in via the external C++ libraries that are available only dynamically.
Comment 26 Iain Sandoe 2019-01-09 12:46:22 UTC
(In reply to Jürgen Reuter from comment #25)
> (In reply to Richard Biener from comment #24)
> > (In reply to Iain Sandoe from comment #23)
> > > (In reply to Jürgen Reuter from comment #22)
> 
> > 
> > Indeed - somehow you didn't get a statically linked executable.  Quoting the
> > full final link command would be interesting.
> 
> The full link commands can be found here, I believe: 
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88750#c14
> 
> Our code generates code for particle physics simulations in the form of
> dynamic libraries that get linked and loaded. For batch clusters, we
> attempted to provide static binaries for these simulations, however, we have
> order 10-15 external libraries that can be linked to our code (which are
> partially mandatory). There are some of them which only exist as dynamic
> libraries, so there our approach cannot result in a purely static binary.
> The static stdc++ library is sucked in via the libtool link mode/flag
> -static-libtool-libs while the dynamic ones are sucked in via the external
> C++ libraries that are available only dynamically.

So .. I appreciate it can be difficult with a sophisticated project.  However, it would seem prudent to try to arrange that you have only one instance of the c++ library.  Imagine creating an object in one instance, and that object somehow finds it's way to be destroyed in a different one.

I've spent some time trying to make it possible to link GCC Darwin projects 'statically', (modulo the libSystem, which must be dynamic) - but that's only going to work if all the project dependent libs are available as convenience libs (or, I suppose, if no used dynamic ones have any external deps other than libSystem).

If that's not possible, then it's most likely better to arrange to do a link -r on everything that can be found as convenience .. and then link the result with -lstdc++.

It might be that it worked before mostly from luck - although I'd still like to have a reference for a known "working" static linked case.  As the c++ library grows, this is only going to be more fragile.
Comment 27 Iain Sandoe 2019-01-09 20:46:01 UTC
JFTR, I did an experiment with a trivial hot/cold partitioned object and ld64 from XCode10.1.

Yes, it complains - but it still publishes the symbol as a weak extern.

====

I looked a bit harder at the symbol resolutions you attached.

* In the working case, the libsupc++ (part of libstdc++) versions of the various new functions/operators are overriding the ones from libc++abi (because the libstdc++.dylib is presented first, I assume).

* In the non-working case, the various new functions/operators are some resolved from the  static instances [static_1.exe:__Zblahblah] - and some from libc++abi.  This is exactly the kind of fragility I was concerned about.

e.g.

dyld: weak bind: static_1.exe:0x10E034738 = static_1.exe:__ZnamRKSt9nothrow_t, *0x10E034738 = 0x10DD4E540
dyld: weak bind: libstdc++.6.dylib:0x112FC0980 = static_1.exe:__ZnamRKSt9nothrow_t, *0x112FC0980 = 0x10DD4E540
dyld: weak bind: libc++abi.dylib:0x7FFF8C76F0A0 = libc++abi.dylib:__ZnamSt11align_val_t, *0x7FFF8C76F0A0 = 0x7FFF578B9C9D
dyld: weak bind: libstdc++.6.dylib:0x112FC0988 = libc++abi.dylib:__ZnamSt11align_val_t, *0x112FC0988 = 0x7FFF578B9C9D
dyld: weak bind: static_1.exe:0x10E034740 = static_1.exe:__Znwm, *0x10E034740 = 0x10DD4E500
dyld: weak bind: liblcio.2.12.dylib:0x1125F0E08 = static_1.exe:__Znwm, *0x1125F0E08 = 0x10DD4E500

and ..

dyld: weak bind: libstdc++.6.dylib:0x112FC0990 = static_1.exe:__Znwm, *0x112FC0990 = 0x10DD4E500
dyld: weak bind: libc++abi.dylib:0x7FFF8C76F0B0 = libc++abi.dylib:__ZnwmSt11align_val_t, *0x7FFF8C76F0B0 = 0x7FFF578B9BE7
dyld: weak bind: libstdc++.6.dylib:0x112FC0998 = libc++abi.dylib:__ZnwmSt11align_val_t, *0x112FC0998 = 0x7FFF578B9BE7
dyld: lazy bind: libgfortran.5.dylib:0x1128D75D0 = libsystem_pthread.dylib:_pthread_key_create, *0x1128D75D0 = 0x7FFF5A38270C

If you look at the working case, these all seem to be consistently resolved from libstdc++.dylib (there's a lot of data, I don't claim to have checked every entry, but a portion).

===

Finally, I don't think that there's been any change to the build of libstdc++ that would actually have changed anything about that particular circumstance (the warning given) in the period mentioned.

** It *is* possible that there's been some change to the system dyld [dynamic linker] during that time as part of an OS upgrade (but not sure if there was an upgrade over that period either) that has tightened up some aspect of the symbol binding - but not sure how to prove/disprove that right now.
Comment 28 Iain Sandoe 2019-01-09 21:11:49 UTC
I wonder what would happen if you add -rdynamic (or -Wl,-export_dynamic) to the main exe in the static link case, perhaps that would ensure that the libstdc++ symbols get resolved from there.
Comment 29 Jürgen Reuter 2019-01-09 23:02:21 UTC
-rdynamic doesn't change anything, and ld doesn't understand -export-dynamic.
I am a bit confused what to do now, as we have a workaround, i.e. using -static
instead of -static-libtool-libs as libtool flag. But in general, the attempt for a completely static binary (at least on MACOSX) doesn't work, so maybe we should retire that feature.
Comment 30 Iain Sandoe 2019-01-09 23:11:08 UTC
well, what I'm trying to achieve is that the exe (with libstdc++ linked in) provides all the symbols.

it's -Wl,-export_dynamic with an underscore, no a hyphen

you might need to add -Wl,-all_load too (to make sure that all archive members are included even if they are not used)
Comment 31 Jürgen Reuter 2019-01-09 23:13:18 UTC
Then I get tons of duplicate symbol lines.
Comment 32 Iain Sandoe 2019-01-10 00:08:44 UTC
(In reply to Jürgen Reuter from comment #31)
> Then I get tons of duplicate symbol lines.

ah well, not so simple then,

then I think the next step is for you to identify the last working revision of  the compiler - we can then analyse what the change was that caused the difference and determine if that's a real regression or just exposing a build system issue.
Comment 33 Jürgen Reuter 2019-01-10 06:41:51 UTC
(In reply to Iain Sandoe from comment #32)
> (In reply to Jürgen Reuter from comment #31)
> > Then I get tons of duplicate symbol lines.
> 
> ah well, not so simple then,
> 
> then I think the next step is for you to identify the last working revision
> of  the compiler - we can then analyse what the change was that caused the
> difference and determine if that's a real regression or just exposing a
> build system issue.

I fear I don't have the capacities to do that right now.
Comment 34 Iain Sandoe 2019-01-10 12:35:18 UTC
(In reply to Jürgen Reuter from comment #33)
> (In reply to Iain Sandoe from comment #32)
> > (In reply to Jürgen Reuter from comment #31)
> > > Then I get tons of duplicate symbol lines.
> > 
> > ah well, not so simple then,

So .. I think things behave as expected .. like so:

$ cat op-new.C
#include <new>

extern int *foo ();

int main() {
 
  int *p2 = new (std::nothrow) int[10];
  int *px = foo ();

  delete[] p2;
  delete[] px;
}

$ cat foo.C
#include <new>

int *foo () {
 
   int *p2 = new (std::nothrow) int[10];
     
  if (p2)
     p2[3] = 10;
  return p2;
}

===

/opt/iains/PR88750/bin/g++ foo.C -shared -o libfoo.dylib

^^^ that makes a shared lib that references the libstdc++.dylib

/opt/iains/PR88750/bin/g++ op-new.C -L. -lfoo -o op-sh

^^ builds a "normal" C++ program that resolves the operator new [] from the shared libstdc++.

/opt/iains/PR88750/bin/gcc -xc++ op-new.C -Wl,-force_load,x86_64-apple-darwin16/libstdc++-v3/src/.libs/libstdc++.a -L. -lfoo -o o-sta

(the path to the static instance is local there only because i patched the operator new [] to print out which version was called.)

^^^ builds a program that uses the "o-sta" as the libstdc++ and resolves both the call in main *and* the call in libfoo.dylib from the instance in o-sta.

=====

So, whatever else might be wrong - it is nothing to do with the warning that ld emits.
Comment 35 Jakub Jelinek 2019-01-11 10:56:35 UTC
(In reply to Jürgen Reuter from comment #33)
> (In reply to Iain Sandoe from comment #32)
> > (In reply to Jürgen Reuter from comment #31)
> > > Then I get tons of duplicate symbol lines.
> > 
> > ah well, not so simple then,
> > 
> > then I think the next step is for you to identify the last working revision
> > of  the compiler - we can then analyse what the change was that caused the
> > difference and determine if that's a real regression or just exposing a
> > build system issue.
> 
> I fear I don't have the capacities to do that right now.

I'm afraid without that there is nothing we can do about it.
While bisection can be time consuming, it doesn't need to be time consuming for you (at least not that much), it can be scripted.
If you have some compiler that works (guess because this PR got marked as [9 Regression] gcc 8.x works for you) and some compiler that doesn't, I see three options for the bisection.
One is build with the "good" compiler and "bad" compiler in two separate directories, verify one works and one doesn't, then make a copy of one of those directories and bisect between the *.o files, start with half "good" ones and half "bad" ones and from the outcome try to narrow it to a particular problematic *.o file.  Another one is similar, but if you have "good" and "bad" options, such as e.g. with -O0 it works and -O2 it doesn't, again bisect object files to find the problematic one.
And the last one is try to bisect the compiler, after verifying "good"/"bad" try to build gcc from half way in between those revisions etc.
The bisection to a particular *.o file is something that might help with the bisection of compilers, you only can bisect the compilation of the problematic TU and on that check what actually changed.
Comment 36 Dominique d'Humieres 2019-03-17 14:11:04 UTC
Any progress? Is it really a gfortran bug?
Comment 37 Jürgen Reuter 2019-03-17 14:30:52 UTC
I'm inclined to advice to close this PR. In principle, it would be good to follow up on this and see which change around Christmas 2018 triggered this, but I fear we don't have the personpower atm.
Comment 38 Jakub Jelinek 2019-03-17 15:31:59 UTC
Closing then.