Re: lto-plugin: mismatch between ld's architecture and GCC's configure --host

[Thomas Schwinge <thomas at codesourcery dot com>]

Hi!

Ping.


On Fri, 25 Oct 2013 09:23:24 +0200, I wrote:
> Ping.  To sum it up, with these patches applied, there are no changes for
> a "regular" build (not using the new configure options).  On the other
> hand, configuring GCC as described, it is possible use the 32-bit x86
> linker for/with a x86_64 build, and get the very same GCC test results as
> when using a x86_64 linker.
> 
> 
> On Mon, 14 Oct 2013 12:28:11 +0200, I wrote:
> > On Sat, 12 Oct 2013 12:20:19 +0200, I wrote:
> > > This is a bit of a weird scenario -- but it is supposed to work fine in
> > > my opinion (but doesn't).
> > > 
> > > I have a GNU toolchain as 32-bit x86 GNU/Linux executables, configured to
> > > to generate code for 32-bit x86 by default, and using -m64 for x86_64.
> > > 
> > > This toolchain I'm using on a x86_64 system (which can execute 32-bit
> > > executables) to build a *native* GCC, that is I'm using the 32-bit
> > > toolchain to build a x86_64 GCC (configuring with CC='gcc -m64' CXX='g++
> > > -m64').  I intend to continue using the 32-bit toolchain's linker, which
> > > also is a 32-bit executable (GNU ld).  That one also defaults to x86
> > > code, but can handle the x86_64 case fine if passed -m elf_x86_64, which
> > > GCC does.
> > > 
> > > That the linker is a 32-bit executable is an implementation detail that
> > > is not important generally: it's a separate process living in its own
> > > address space.  However it becomes relevant in the case of linker
> > > plugins: the native x86_64 GCC that I'm building also builds a x86_64
> > > lto-plugin, which the 32-bit ld cannot load:
> > > 
> > >     $ gcc/xgcc -B[...] [...]/gcc.c-torture/execute/ieee/20000320-1.c [...] -flto [...]
> > >     [...]/ld: [...]/gcc/liblto_plugin.so: error loading plugin: [...]/gcc/liblto_plugin.so: wrong ELF class: ELFCLASS64
> > >     collect2: error: ld returned 1 exit status
> > > 
> > > So, aside from building a 64-bit ld (which is the "lame" alternative), I
> > > now need to teach GCC's build system that the lto-plugin may need special
> > > configuration: CC='gcc -m32' -- and [...] its own build of libiberty,
> > > too [...]
> > 
> > > Instead of auto-detecting the linker's
> > > architecture (and then, what to do with that information?), I intend to
> > > make this a manual process (so, some new top-level configure
> > > argument(s)).  Adding yet another set of {...,CC,...}_FOR_[something] is
> > > probably overkill -- I'll try to find something simpler.
> > > 
> > > Any comments on this scenario?
> > 
> > Here are the patches.  Unless the new option is exercised, there are no
> > effects on a native x86_64 GNU/Linux bootstrap build (the build trees'
> > *.o files are identical, as are the test results).  OK to commit?
> 
> > Allow overriding the libiberty used for building the LTO plugin.
> > 
> > 	lto-plugin/
> > 	* configure.ac (--with-libiberty): New configure option.
> > 	* configure: Regenerate.
> > 	* Makefile.am (libiberty, libiberty_pic): New variables.
> > 	(liblto_plugin_la_LIBADD, liblto_plugin_la_LDFLAGS)
> > 	(liblto_plugin_la_DEPENDENCIES): Use them.
> > 	* Makefile.in: Regenerate.
> > ---
> >  lto-plugin/Makefile.am  | 20 +++++++++++---------
> >  lto-plugin/Makefile.in  | 22 ++++++++++++----------
> >  lto-plugin/configure    | 17 +++++++++++++++--
> >  lto-plugin/configure.ac |  5 +++++
> >  4 files changed, 43 insertions(+), 21 deletions(-)
> > 
> > diff --git lto-plugin/Makefile.am lto-plugin/Makefile.am
> > index b24015e..8b7bb54 100644
> > --- lto-plugin/Makefile.am
> > +++ lto-plugin/Makefile.am
> > @@ -15,17 +15,19 @@ libexecsub_LTLIBRARIES = liblto_plugin.la
> >  gcc_build_dir = ../$(host_subdir)/gcc
> >  in_gcc_libs = $(foreach lib, $(libexecsub_LTLIBRARIES), $(gcc_build_dir)/$(lib))
> >  
> > -# Can be removed when libiberty becomes a normal convenience library
> > -Wc=-Wc,
> > -
> >  liblto_plugin_la_SOURCES = lto-plugin.c
> > +# Note that we intentionally override the bindir supplied by ACX_LT_HOST_FLAGS.
> > +liblto_plugin_la_LDFLAGS = $(lt_host_flags) -module -bindir $(libexecsubdir)
> > +# Can be simplified when libiberty becomes a normal convenience library.
> > +libiberty=$(with_libiberty)/libiberty.a
> > +libiberty_pic=$(with_libiberty)/pic/libiberty.a
> > +Wc=-Wc,
> >  liblto_plugin_la_LIBADD = \
> > -	$(if $(wildcard ../libiberty/pic/libiberty.a),$(Wc)../libiberty/pic/libiberty.a,)
> > -# Note that we intentionally override the bindir supplied by ACX_LT_HOST_FLAGS
> > -liblto_plugin_la_LDFLAGS = $(lt_host_flags) -module -bindir $(libexecsubdir) \
> > -	$(if $(wildcard ../libiberty/pic/libiberty.a),,-Wc,../libiberty/libiberty.a)
> > -liblto_plugin_la_DEPENDENCIES = $(if $(wildcard \
> > -	../libiberty/pic/libiberty.a),../libiberty/pic/libiberty.a,)
> > +	$(if $(wildcard $(libiberty_pic)),$(Wc)$(libiberty_pic),)
> > +liblto_plugin_la_LDFLAGS += \
> > +	$(if $(wildcard $(libiberty_pic)),,-Wc,$(libiberty))
> > +liblto_plugin_la_DEPENDENCIES = \
> > +	$(if $(wildcard $(libiberty_pic)),$(libiberty_pic),)
> >  
> >  all-local: $(in_gcc_libs)
> >  
> > diff --git lto-plugin/configure.ac lto-plugin/configure.ac
> > index 9a418d2..b73fabb 100644
> > --- lto-plugin/configure.ac
> > +++ lto-plugin/configure.ac
> > @@ -4,6 +4,11 @@ AC_CANONICAL_SYSTEM
> >  GCC_TOPLEV_SUBDIRS
> >  AM_INIT_AUTOMAKE([foreign no-dist])
> >  AM_MAINTAINER_MODE
> > +AC_ARG_WITH(libiberty,
> > +  [AS_HELP_STRING([--with-libiberty=PATH],
> > +    [specify the directory where to find libiberty [../libiberty]])],
> > +  [], with_libiberty=../libiberty)
> > +AC_SUBST(with_libiberty)
> >  AC_PROG_CC
> >  AC_SYS_LARGEFILE
> >  ACX_PROG_CC_WARNING_OPTS([-Wall], [ac_lto_plugin_warn_cflags])
> 
> > Allow for overriding a module's srcdir.
> > 
> > 	* Makefile.tpl (configure-[+prefix+][+module+])
> > 	(configure-stage[+id+]-[+prefix+][+module+]): Allow for
> > 	overriding a module's srcdir.
> > 	* Makefile.in: Regenerate.
> > 
> > ---
> >  Makefile.in  | 2141 ++++++++++++++++++++++++++++++----------------------------
> >  Makefile.tpl |   21 +-
> >  2 files changed, 1132 insertions(+), 1030 deletions(-)
> > 
> > diff --git Makefile.tpl Makefile.tpl
> > index 2695a2b..36d480e 100644
> > --- Makefile.tpl
> > +++ Makefile.tpl
> > @@ -1000,12 +1000,14 @@ configure-[+prefix+][+module+]: [+ IF bootstrap +][+ ELSE +]
> >  	  *) topdir=`echo [+subdir+]/[+module+]/ | \
> >  		sed -e 's,\./,,g' -e 's,[^/]*/,../,g' `$(srcdir) ;; \
> >  	esac; \
> > -	srcdiroption="--srcdir=$${topdir}/[+module+]"; \
> > -	libsrcdir="$$s/[+module+]"; \
> > +	module_srcdir=[+ IF module_srcdir +][+module_srcdir+][+
> > +	ELSE module_srcdir +][+module+][+ ENDIF module_srcdir +]; \
> >  	[+ IF no-config-site +]rm -f no-such-file || : ; \
> > -	CONFIG_SITE=no-such-file [+ ENDIF +]$(SHELL) $${libsrcdir}/configure \
> > +	CONFIG_SITE=no-such-file [+ ENDIF +]$(SHELL) \
> > +	  $$s/$$module_srcdir/configure \
> > +	  --srcdir=$${topdir}/$$module_srcdir \
> >  	  [+args+] --build=${build_alias} --host=[+host_alias+] \
> > -	  --target=[+target_alias+] $${srcdiroption} [+extra_configure_flags+] \
> > +	  --target=[+target_alias+] [+extra_configure_flags+] \
> >  	  || exit 1
> >  @endif [+prefix+][+module+]
> >  
> > @@ -1053,12 +1055,13 @@ configure-stage[+id+]-[+prefix+][+module+]:
> >  	  *) topdir=`echo [+subdir+]/[+module+]/ | \
> >  		sed -e 's,\./,,g' -e 's,[^/]*/,../,g' `$(srcdir) ;; \
> >  	esac; \
> > -	srcdiroption="--srcdir=$${topdir}/[+module+]"; \
> > -	libsrcdir="$$s/[+module+]"; \
> > -	$(SHELL) $${libsrcdir}/configure \
> > +	module_srcdir=[+ IF module_srcdir +][+module_srcdir+][+
> > +	ELSE module_srcdir +][+module+][+ ENDIF module_srcdir +]; \
> > +	$(SHELL) $$s/$$module_srcdir/configure \
> > +	  --srcdir=$${topdir}/$$module_srcdir \
> >  	  [+args+] --build=${build_alias} --host=[+host_alias+] \
> > -	  --target=[+target_alias+] $${srcdiroption} [+ IF prev +]\
> > -	  --with-build-libsubdir=$(HOST_SUBDIR) [+ ENDIF prev +]\
> > +	  --target=[+target_alias+] \
> > +	  [+ IF prev +] --with-build-libsubdir=$(HOST_SUBDIR) [+ ENDIF prev +] \
> >  	  $(STAGE[+id+]_CONFIGURE_FLAGS)[+ IF extra_configure_flags +] \
> >  	  [+extra_configure_flags+][+ ENDIF extra_configure_flags +]
> >  @endif [+prefix+][+module+]-bootstrap
> 
> > Non-host system configuration for linker plugins.
> > 
> > 	* configure.ac (--enable-linker-plugin-flags)
> > 	(--enable-linker-plugin-configure-flags): New flags.
> > 	(configdirs): Conditionally add libiberty-linker-plugin.
> > 	* configure: Regenerate.
> > 	* Makefile.def (host_modules): Add libiberty-linker-plugin.
> > 	(host_modules) <lto-plugin>: Pay attention to
> > 	@extra_linker_plugin_flags@ and
> > 	@extra_linker_plugin_configure_flags@.
> > 	(all-lto-plugin): Also depend on all-libiberty-linker-plugin.
> > 	* Makefile.in: Regenerate.
> > 	gcc/
> > 	* doc/install.texi (--enable-linker-plugin-flags)
> > 	(--enable-linker-plugin-configure-flags): Document new flags.
> > ---
> >  Makefile.def         |   14 +-
> >  Makefile.in          | 1051 ++++++++++++++++++++++++++++++++++++++++++++++++--
> >  configure            |   42 ++
> >  configure.ac         |   28 ++
> >  gcc/doc/install.texi |   19 +
> >  5 files changed, 1117 insertions(+), 37 deletions(-)
> > 
> > diff --git Makefile.def Makefile.def
> > index 3ba1a5b..ea48792 100644
> > --- Makefile.def
> > +++ Makefile.def
> > @@ -85,6 +85,14 @@ host_modules= { module= libdecnumber; bootstrap=true; };
> >  host_modules= { module= libgui; };
> >  host_modules= { module= libiberty; bootstrap=true;
> >  	        extra_configure_flags='@extra_host_libiberty_configure_flags@';};
> > +// Linker plugins may need their own build of libiberty; see
> > +// gcc/doc/install.texi.  We take care that this build of libiberty doesn't get
> > +// installed.  It's a helper library for linker plugins, so we pay attention to
> > +// @extra_linker_plugin_flags@ and @extra_linker_plugin_configure_flags@.
> > +host_modules= { module= libiberty-linker-plugin; bootstrap=true;
> > +		module_srcdir=libiberty;
> > +	        extra_configure_flags='@extra_host_libiberty_configure_flags@ --disable-install-libiberty @extra_linker_plugin_flags@ @extra_linker_plugin_configure_flags@';
> > +		extra_make_flags='@extra_linker_plugin_flags@'; };
> >  // We abuse missing to avoid installing anything for libiconv.
> >  host_modules= { module= libiconv;
> >  		extra_configure_flags='--disable-shared';
> > @@ -111,7 +119,8 @@ host_modules= { module= libtermcap; no_check=true;
> >  host_modules= { module= utils; no_check=true; };
> >  host_modules= { module= gnattools; };
> >  host_modules= { module= lto-plugin; bootstrap=true;
> > -		extra_configure_flags=--enable-shared; };
> > +		extra_configure_flags='--enable-shared @extra_linker_plugin_flags@ @extra_linker_plugin_configure_flags@';
> > +		extra_make_flags='@extra_linker_plugin_flags@'; };
> >  
> >  target_modules = { module= libstdc++-v3;
> >  		   bootstrap=true;
> > @@ -336,7 +345,10 @@ dependencies = { module=all-fixincludes; on=all-libiberty; };
> >  
> >  dependencies = { module=all-gnattools; on=all-target-libada; };
> >  
> > +// Depending on the specific configuration, the LTO plugin will either use the
> > +// generic libiberty build or the specific build for linker plugins.
> >  dependencies = { module=all-lto-plugin; on=all-libiberty; };
> > +dependencies = { module=all-lto-plugin; on=all-libiberty-linker-plugin; };
> >  
> >  dependencies = { module=all-utils; on=all-libiberty; };
> >  
> > diff --git configure.ac configure.ac
> > index b5caebb..128c60b 100644
> > --- configure.ac
> > +++ configure.ac
> > @@ -1731,6 +1731,19 @@ ACX_ELF_TARGET_IFELSE([# ELF platforms build the lto-plugin always.
> >    esac
> >  ])
> >  
> > +AC_ARG_ENABLE(linker-plugin-flags,
> > +  [AS_HELP_STRING([[--enable-linker-plugin-flags=FLAGS]],
> > +    [additional flags for configuring and building linker plugins @<:@none@:>@])],
> > +  extra_linker_plugin_flags=$enableval,
> > +  extra_linker_plugin_flags=)
> > +AC_SUBST(extra_linker_plugin_flags)
> > +AC_ARG_ENABLE(linker-plugin-configure-flags,
> > +  [AS_HELP_STRING([[--enable-linker-plugin-configure-flags=FLAGS]],
> > +    [additional flags for configuring linker plugins @<:@none@:>@])],
> > +  extra_linker_plugin_configure_flags=$enableval,
> > +  extra_linker_plugin_configure_flags=)
> > +AC_SUBST(extra_linker_plugin_configure_flags)
> > +
> >  
> >  # By default, C and C++ are the only stage 1 languages.
> >  stage1_languages=,c,
> > @@ -2133,6 +2146,21 @@ for i in ${target_configdirs_all} ; do
> >    fi
> >  done
> >  
> > +# libiberty-linker-plugin is special: it doesn't have its own source directory,
> > +# so we have to add it after the preceding checks.
> > +if test x"$extra_linker_plugin_flags$extra_linker_plugin_configure_flags" != x
> > +then
> > +  case " $configdirs " in
> > +    *" libiberty "*)
> > +      # If we can build libiberty, we can also build libiberty-linker-plugin.
> > +      configdirs="$configdirs libiberty-linker-plugin"
> > +      extra_linker_plugin_configure_flags="$extra_linker_plugin_configure_flags \
> > +        --with-libiberty=../libiberty-linker-plugin";;
> > +    *)
> > +      AC_MSG_ERROR([libiberty missing]);;
> > +  esac
> > +fi
> > +
> >  # Produce a warning message for the subdirs we can't configure.
> >  # This isn't especially interesting in the Cygnus tree, but in the individual
> >  # FSF releases, it's important to let people know when their machine isn't
> > diff --git gcc/doc/install.texi gcc/doc/install.texi
> > index 7be8e5a..74cf660 100644
> > --- gcc/doc/install.texi
> > +++ gcc/doc/install.texi
> > @@ -1746,6 +1746,25 @@ GLIBC 2.11 or above, otherwise disabled.
> >  Enable support for link-time optimization (LTO).  This is enabled by
> >  default, and may be disabled using @option{--disable-lto}.
> >  
> > +@item --enable-linker-plugin-flags=FLAGS
> > +@itemx --enable-linker-plugin-configure-flags=FLAGS
> > +By default, linker plugins (such as the LTO plugin) are built for the
> > +host system architecture.  For the case that the linker has a
> > +different (but run-time compatible) architecture, these flags can be
> > +specified to build plugins that are compatible to the linker.  For
> > +example, if you are building GCC for a 64-bit x86_64
> > +(@samp{x86_64-unknown-linux-gnu}) host system, but have a 32-bit x86
> > +GNU/Linux (@samp{i686-pc-linux-gnu}) linker executable (which is
> > +executable on the former system), you can configure GCC as follows for
> > +getting compatible linker plugins:
> > +
> > +@smallexample
> > +% @var{srcdir}/configure \
> > +    --host=x86_64-unknown-linux-gnu \
> > +    --enable-linker-plugin-flags='CC=gcc\ -m32' \
> > +    --enable-linker-plugin-configure-flags=--host=i686-pc-linux-gnu
> > +@end smallexample
> > +
> >  @item --with-plugin-ld=@var{pathname}
> >  Enable an alternate linker to be used at link-time optimization (LTO)
> >  link time when @option{-fuse-linker-plugin} is enabled.
> 
> > GNU ld can use linker plugins, too.
> > 
> > 	gcc/
> > 	* doc/sourcebuild.texi (Top Level) <lto-plugin>: GNU ld can use
> > 	linker plugins, too.
> > 
> > ---
> >  gcc/doc/sourcebuild.texi | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git gcc/doc/sourcebuild.texi gcc/doc/sourcebuild.texi
> > index 90bd0bd..23ff04a 100644
> > --- gcc/doc/sourcebuild.texi
> > +++ gcc/doc/sourcebuild.texi
> > @@ -120,7 +120,7 @@ The Stack protector runtime library.
> >  The C++ runtime library.
> >  
> >  @item lto-plugin
> > -Plugin used by @command{gold} if link-time optimizations are enabled.
> > +Plugin used by the linker if link-time optimizations are enabled.
> >  
> >  @item maintainer-scripts
> >  Scripts used by the @code{gccadmin} account on @code{gcc.gnu.org}.
> 
> > Fix typo.
> > 
> > 	* Makefile.tpl: Fix typo.
> > 	* Makefile.in: Regenerate.
> > 
> > ---
> >  Makefile.in  | 2 +-
> >  Makefile.tpl | 2 +-
> >  2 files changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git Makefile.in Makefile.in
> > index a13771d..67d8cfb 100644
> > --- Makefile.in
> > +++ Makefile.in
> > @@ -166,7 +166,7 @@ BUILD_EXPORTS = \
> >  	WINDMC="$(WINDMC_FOR_BUILD)"; export WINDMC;
> >  
> >  # These variables must be set on the make command line for directories
> > -# built for the build system to override those in BASE_FLAGS_TO_PASSS.
> > +# built for the build system to override those in BASE_FLAGS_TO_PASS.
> >  EXTRA_BUILD_FLAGS = \
> >  	CFLAGS="$(CFLAGS_FOR_BUILD)" \
> >  	LDFLAGS="$(LDFLAGS_FOR_BUILD)"
> > diff --git Makefile.tpl Makefile.tpl
> > index 3233a78..2695a2b 100644
> > --- Makefile.tpl
> > +++ Makefile.tpl
> > @@ -169,7 +169,7 @@ BUILD_EXPORTS = \
> >  	WINDMC="$(WINDMC_FOR_BUILD)"; export WINDMC;
> >  
> >  # These variables must be set on the make command line for directories
> > -# built for the build system to override those in BASE_FLAGS_TO_PASSS.
> > +# built for the build system to override those in BASE_FLAGS_TO_PASS.
> >  EXTRA_BUILD_FLAGS = \
> >  	CFLAGS="$(CFLAGS_FOR_BUILD)" \
> >  	LDFLAGS="$(LDFLAGS_FOR_BUILD)"


GrÃÃe,
 Thomas

Attachment: pgpbcrAmEPH6S.pgp
Description: PGP signature