Bug 63610 - Fixing the libtool bug for Yosemite (darwin14)
Summary: Fixing the libtool bug for Yosemite (darwin14)
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: target (show other bugs)
Version: 4.9.1
: P3 normal
Target Milestone: 5.0
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-10-21 18:35 UTC by Volker Braun
Modified: 2015-08-12 02:30 UTC (History)
4 users (show)

See Also:
Host: x86_64-apple-darwin14
Target: x86_64-apple-darwin14
Build: x86_64-apple-darwin14
Known to work:
Known to fail:
Last reconfirmed: 2014-11-11 00:00:00


Attachments
Log of the MPIR compilation ending in the testsuite failure (26.57 KB, text/plain)
2014-10-21 18:36 UTC, Volker Braun
Details
gcc -v output (273 bytes, text/plain)
2014-10-21 18:36 UTC, Volker Braun
Details
gdb log of the failing testcase (1.56 KB, text/plain)
2014-10-21 18:37 UTC, Volker Braun
Details
convenience patch (780 bytes, patch)
2014-10-27 12:43 UTC, Jack Howarth
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Volker Braun 2014-10-21 18:35:43 UTC
Trying to bulid gcc on OSX 10.10 with the Apple command line tools. So I first compile MPIR/MPFR/MPC with clang, then bootstrap gcc. This works so far (the patch from PR target/61407 for dealing with the version parsing is applied). But the newly-built gcc can't successfully compile MPIR, and I'm getting the testsuite failures below. The failure is only present if I both enable C++ (--enable-cxx in the MPIR configure), build a shared library, and set optimization to -O0 or -O1 (whereas -O2 works).

Steps to reproduce (in the MPIR 2.6.0 source directory):

$ export CFLAGS=-O0
$ ./configure --enable-cxx --disable-static
$ make
$ make check
[...]
make  check-TESTS
/bin/sh: line 1: 59465 Segmentation fault: 11  "$tst" > t-bswap.log-t 2>&1
FAIL: t-bswap
PASS: t-constants
PASS: t-count_zeros
PASS: t-gmpmax
PASS: t-hightomask
PASS: t-modlinv
PASS: t-parity
/bin/sh: line 1: 59648 Segmentation fault: 11  "$tst" > t-popc.log-t 2>&1
FAIL: t-popc
PASS: t-sub
=============================================================
2 of 9 tests failed
See tests/test-suite.log
Please report to http://groups.google.co.uk/group/mpir-devel/
=============================================================
make[5]: *** [test-suite.log] Error 1
make[4]: *** [check-TESTS] Error 2
make[3]: *** [check-am] Error 2
make[2]: *** [check-recursive] Error 1
make[1]: *** [check-recursive] Error 1
make: *** [check] Error 2

Apart from the segfault, there is nothing in the log (full log attached) that indicates a failure. It seems that lower optimization leaves something in the binary that interacts badly with C++ support, but I don't have a better testcase. It is not just in MPIR, I get similar looking segfaults in other shared libraries when C++ is involved. Others have been able to reproduce the same testsuite failure on OSX. Works fine on Linux x86_64 with same versions of everything (except that gcc is used to bootstrap, of course).

Stack backtrace from the t-popc test (full gdb log attached):

#0  0x00007fff86172432 in __block_descriptor_tmp227 () from /usr/lib/system/libdyld.dylib
#1  0x00007fff5fbffa80 in ?? ()
#2  0x00000001000ca0d0 in ?? () from /Users/vbraun/Code/mpir-2.6.0/.libs/libmpir.11.dylib
#3  0x000000000000011d in ?? ()
#4  0x00000001000223c1 in __gmp_randget_mt (rstate=0x7fff5fbffb60, dest=0x7fff5fbffab0, nbits=32)
    at randmt.c:256
#5  0x000000010006d81d in ?? () from /Users/vbraun/Code/mpir-2.6.0/.libs/libmpir.11.dylib
#6  0x000000010006d71c in ?? () from /Users/vbraun/Code/mpir-2.6.0/.libs/libmpir.11.dylib
#7  0x0000000100000f8c in main () at t-popc.c:60
(gdb) frame 4
#4  0x00000001000223c1 in __gmp_randget_mt (rstate=0x7fff5fbffb60, dest=0x7fff5fbffab0, nbits=32)
    at randmt.c:256
256		  NEXT_RANDOM;

The NEXT_RANDOM macro is just the next step of MPIR's Mersenne twister, it only does elementary bit operations on the state variable. It also only fails after the 70th iteration of the outermost loop. Successive runs always fail in the same __block_descriptor_tmp227. The other failing testcase t-bswap also segfaults in the Mersenne twister.
Comment 1 Volker Braun 2014-10-21 18:36:29 UTC
Created attachment 33770 [details]
Log of the MPIR compilation ending in the testsuite failure
Comment 2 Volker Braun 2014-10-21 18:36:56 UTC
Created attachment 33771 [details]
gcc -v output
Comment 3 Volker Braun 2014-10-21 18:37:54 UTC
Created attachment 33773 [details]
gdb log of the failing testcase
Comment 4 Volker Braun 2014-10-23 13:22:14 UTC
I got my hands on a gcc 4.9.1 binary build made on OSX 10.9 and this one works. Diffing the log shows that this is due to different linker flags, my binary build calls 

libtool: link: gcc -std=gnu99 -dynamiclib -Wl,-undefined -Wl,dynamic_lookup -o .libs/libmpir.11.dylib  [...]

whereas the Yosemite-bootstrapped gcc calls

libtool: link: gcc -std=gnu99 -dynamiclib -Wl,-flat_namespace -Wl,-undefined -Wl,suppress -o .libs/libmpir.11.dylib [...]

Relinking with the previous linker flags fixes the segfault. This is most likely due to incorrect aclocal.m4 case check

      case ${MACOSX_DEPLOYMENT_TARGET-10.0},$host in
        10.0,*86*-darwin8*|10.0,*-darwin[[91]]*)
          _lt_dar_allow_undefined='${wl}-undefined ${wl}dynamic_lookup' ;;
        10.[[012]]*)
          _lt_dar_allow_undefined='${wl}-flat_namespace ${wl}-undefined ${wl}suppress' ;;

not taking 10.10 properly into account.
Comment 5 Volker Braun 2014-10-23 13:56:04 UTC
Libtool upstream patch:

​http://lists.gnu.org/archive/html/libtool-patches/2014-09/msg00002.html
Comment 6 Jack Howarth 2014-10-27 12:43:59 UTC
Created attachment 33817 [details]
convenience patch
Comment 7 Jack Howarth 2014-10-27 12:48:23 UTC
I can confirm that there are many regressions (particularly in the fortran test suite) on Yosemite due to the libtool bug which causes shared libraries to be linked as if the target was Puma. The attached convenience patch solves the issue (which should only be triggered if MACOSX_DEPLOYMENT_TARGET is being set). The fix is of the form...

Index: gcc-4.9.1/gcc/configure
===================================================================
--- gcc-4.9.1.orig/gcc/configure
+++ gcc-4.9.1/gcc/configure
@@ -14415,7 +14415,7 @@ $as_echo "$lt_cv_ld_force_load" >&6; }
       case ${MACOSX_DEPLOYMENT_TARGET-10.0},$host in
        10.0,*86*-darwin8*|10.0,*-darwin[91]*)
          _lt_dar_allow_undefined='${wl}-undefined ${wl}dynamic_lookup' ;;
-       10.[012]*)
+       10.[012][,.]*)
          _lt_dar_allow_undefined='${wl}-flat_namespace ${wl}-undefined ${wl}suppress' ;;
        10.*)
          _lt_dar_allow_undefined='${wl}-undefined ${wl}dynamic_lookup' ;;
Comment 8 Volker Braun 2014-10-27 13:06:53 UTC
Another workaround is to set MACOSX_DEPLOYMENT_TARGET=10.9

Libtool-2.4.3 will have the fix, gcc (and everybody else) should just reconfigure when its out.

http://lists.gnu.org/archive/html/libtool/2014-10/msg00001.html
Comment 9 Jack Howarth 2014-10-27 13:31:08 UTC
Alternatively, you can just make sure you don't set MACOSX_DEPLOYMENT_TARGET in your shell. 

Also mote that gmp, mpfr and mpc also suffer from this bug and, if built on 10.10, should either have their configure scripts patched or regenerated with the fixed libtool.
Comment 10 Jack Howarth 2014-11-07 12:45:42 UTC
(In reply to Volker Braun from comment #8)
> Another workaround is to set MACOSX_DEPLOYMENT_TARGET=10.9
> 
> Libtool-2.4.3 will have the fix, gcc (and everybody else) should just
> reconfigure when its out.
> 
> http://lists.gnu.org/archive/html/libtool/2014-10/msg00001.html

Libtool 2.4.3 was released on Oct 27th. There should be a 2.4.4 released with a week, http://lists.gnu.org/archive/html/bug-libtool/2014-10/msg00009.html, to eliminate http://lists.gnu.org/archive/html/bug-libtool/2014-10/msg00006.html. Upstream contends that this bug is limited to the libtool installation and isn't transmitted beyond.
Comment 11 Francois-Xavier Coudert 2014-11-11 09:15:34 UTC
(In reply to howarth from comment #7)
> I can confirm that there are many regressions (particularly in the fortran
> test suite) on Yosemite due to the libtool bug

Patch submitted to fix the configure files in GCC tree: https://gcc.gnu.org/ml/gcc-patches/2014-11/msg00930.html

But it doesn't seem to make much difference in the test results: https://gcc.gnu.org/ml/gcc-testresults/2014-11/msg01024.html
Comment 12 Francois-Xavier Coudert 2014-11-11 20:25:50 UTC
Author: fxcoudert
Date: Tue Nov 11 20:25:18 2014
New Revision: 217366

URL: https://gcc.gnu.org/viewcvs?rev=217366&root=gcc&view=rev
Log:
toplevel:
2014-11-11  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	PR target/63610
	* libtool.m4: Fix globbing of darwin versions.

boehm-gc/
2014-11-11  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	PR target/63610
	* configure: Regenerate.

gcc/
2014-11-11  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	PR target/63610
	* configure: Regenerate.

libatomic/
2014-11-11  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	PR target/63610
	* configure: Regenerate.

libbacktrace/
2014-11-11  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	PR target/63610
	* configure: Regenerate.

libcc1/
2014-11-11  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	PR target/63610
	* configure: Regenerate.

libcilkrts/
2014-11-11  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	PR target/63610
	* configure: Regenerate.

libffi/
2014-11-11  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	PR target/63610
	* configure: Regenerate.

libgfortran/
2014-11-11  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	PR target/63610
	* configure: Regenerate.

libgomp/
2014-11-11  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	PR target/63610
	* configure: Regenerate.

libitm/
2014-11-11  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	PR target/63610
	* configure: Regenerate.

libobjc/
2014-11-11  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	PR target/63610
	* configure: Regenerate.

libquadmath/
2014-11-11  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	PR target/63610
	* configure: Regenerate.

libsanitizer/
2014-11-11  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	PR target/63610
	* configure: Regenerate.

libssp/
2014-11-11  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	PR target/63610
	* configure: Regenerate.

libstdc++-v3/
2014-11-11  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	PR target/63610
	* configure: Regenerate.

libvtv/
2014-11-11  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	PR target/63610
	* configure: Regenerate.

lto-plugin/
2014-11-11  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	PR target/63610
	* configure: Regenerate.

libjava/
2014-11-11  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	PR target/63610
	* configure: Regenerate.

libjava/classpath/
2014-11-11  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	PR target/63610
	* configure: Regenerate.

zlib/
2014-11-11  Francois-Xavier Coudert  <fxcoudert@gcc.gnu.org>

	PR target/63610
	* configure: Regenerate.

Modified:
    trunk/ChangeLog
    trunk/boehm-gc/ChangeLog
    trunk/boehm-gc/configure
    trunk/gcc/ChangeLog
    trunk/gcc/configure
    trunk/libatomic/ChangeLog
    trunk/libatomic/configure
    trunk/libbacktrace/ChangeLog
    trunk/libbacktrace/configure
    trunk/libcc1/ChangeLog
    trunk/libcc1/configure
    trunk/libcilkrts/ChangeLog
    trunk/libcilkrts/configure
    trunk/libffi/ChangeLog
    trunk/libffi/configure
    trunk/libgfortran/ChangeLog
    trunk/libgfortran/configure
    trunk/libgomp/ChangeLog
    trunk/libgomp/configure
    trunk/libitm/ChangeLog
    trunk/libitm/configure
    trunk/libjava/ChangeLog
    trunk/libjava/classpath/ChangeLog.gcj
    trunk/libjava/classpath/configure
    trunk/libjava/configure
    trunk/libobjc/ChangeLog
    trunk/libobjc/configure
    trunk/libquadmath/ChangeLog
    trunk/libquadmath/configure
    trunk/libsanitizer/ChangeLog
    trunk/libsanitizer/configure
    trunk/libssp/ChangeLog
    trunk/libssp/configure
    trunk/libstdc++-v3/ChangeLog
    trunk/libstdc++-v3/configure
    trunk/libtool.m4
    trunk/libvtv/ChangeLog
    trunk/libvtv/configure
    trunk/lto-plugin/ChangeLog
    trunk/lto-plugin/configure
    trunk/zlib/ChangeLog.gcj
    trunk/zlib/configure
Comment 13 Francois-Xavier Coudert 2014-11-12 22:44:12 UTC
Fixed on trunk. I don't see the need to back port to 4.9, since I don't see many test suite failures on that branch. If you have evidence that a back port would help, please reopen and add the new info.
Comment 14 Jack Howarth 2014-11-12 22:59:38 UTC
(In reply to Francois-Xavier Coudert from comment #13)
> Fixed on trunk. I don't see the need to back port to 4.9, since I don't see
> many test suite failures on that branch. If you have evidence that a back
> port would help, please reopen and add the new info.

   You aren't seeing regressions because you're not setting MACOSX_DEPLOYMENT_TARGET  during the gcc build. It took me days to puzzle that out why a standalone build of gcc 4.9.2 on darwin14 produced clean test suite results but one under the fink packaging system, which sets MACOSX_DEPLOYMENT_TARGET, produced a huge number of regressions in the fortran test suite.
   So you definitely do want to back port the configure/libtool.m4 fixes for 4.9.3.
Comment 15 Francois-Xavier Coudert 2014-11-14 08:36:51 UTC
(In reply to howarth from comment #14)
> You aren't seeing regressions because you're not setting
> MACOSX_DEPLOYMENT_TARGET  during the gcc build.

Oh, so that's it! Thanks for puzzling it out.

I'll ask RM for permission to backport to all active branches (4.8 and 4.9). Before I do that, can you confirm that the patch works as you expect on trunk, with MACOSX_DEPLOYMENT_TARGET set? Just to be extra sure.
Comment 16 Iain Sandoe 2014-11-14 08:52:02 UTC
Hmm the patch for __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ has been applied to 4.9 (AFAICT), although not to 4.8,

Jack, what does your 4.9 compiler give for:

gcc -dM -E -x c /dev/null |grep ENVIRO

?

If the __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ is being set correctly, what is causing the need for MACOSX_DEPLOYMENT_TARGET?
Comment 17 Francois-Xavier Coudert 2014-11-14 09:16:02 UTC
(In reply to Iain Sandoe from comment #16)
> Hmm the patch for __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ has been
> applied to 4.9 (AFAICT)

Yes it has.


> although not to 4.8

You're right. Provided that the patches are simple, maybe it's good enough to document them (as I have in the __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ patch for 4.8) and let distributors apply them.
Comment 18 Jack Howarth 2014-11-14 11:19:28 UTC
(In reply to Francois-Xavier Coudert from comment #17)
> (In reply to Iain Sandoe from comment #16)
> > Hmm the patch for __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ has been
> > applied to 4.9 (AFAICT)
> 
> Yes it has.
> 
> 
> > although not to 4.8
> 
> You're right. Provided that the patches are simple, maybe it's good enough
> to document them (as I have in the
> __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ patch for 4.8) and let
> distributors apply them.

There is nothing in the gcc build that *needs* to have MACOSX_DEPLOYMENT_TARGET set, however some packaging environments, like fink and MacPorts, happen to set MACOSX_DEPLOYMENT_TARGET by default and thus triggers the configure/libtool.m4 bug...

https://trac.macports.org/ticket/44596#comment:35

The Apple programmers told of the libtool.m4 bug so far all are of the view that configure/libtool.m4 shouldn't really be defaulting to "-undefined dynamic_lookup" as that can mask undefined symbols in release code (but should be default "-undefined error").

https://trac.macports.org/ticket/44596#comment:48

However that is a really libtool bug.