Re: [PATCH] Avoid invoking ranlib on libbackend.a

[Richard Biener <richard dot guenther at gmail dot com>]

On Tue, Jul 19, 2016 at 2:39 AM, Patrick Palka <patrick@parcs.ath.cx> wrote:
> On Mon, 18 Jul 2016, Segher Boessenkool wrote:
>
>> On Mon, Jul 18, 2016 at 06:35:11AM -0500, Segher Boessenkool wrote:
>> > Or, if using GNU ar, you can even use -S, if that helps (after testing
>> > for it in configure, of course).
>>
>> I meant -T.  Some day I will learn how to type, promise!
>
> According to the documentation of GNU ar,
>
>   "gnu ar can optionally create a thin archive, which contains a symbol
>   index and references to the original copies of the member files of the
>   archive. This is useful for building libraries for use within a local
>   build tree, where the relocatable objects are expected to remain
>   available, and copying the contents of each object would only waste time
>   and space."
>
> Since the objects which libbackend.a is composed of remain available
> throughout the build process I think it should be safe to make
> libbackend.a a thin archive.
>
> So here's a patch which builds libbackend.a as a thin archive if the
> toolchain supports it.  The time it takes to rebuild a
> --disable-bootstrap tree after touching a single source file is now 7.5s
> instead of 35+s -- a much better speedup than when simply eliding the
> call to ranlib since the archive is now 1-5MB in size instead of 450MB.
>
> Instead of changing AR_FLAGS, only the invocation of ar on libbackend.a
> is changed because that is by far the largest archive (by a factor of
> 20x) and it seems less risky this way.
>
> One thing that was not clear to me is whether the object file paths
> stored in a thin archive are relative or absolute paths.  If they are
> absolute paths then that would be a problem due to how the build system
> moves build directories in between stages (gcc/ -> prev-gcc/ etc).  But
> it looks like the object file paths are relative to the location of the
> archive which is compatible.
>
> Bootstrapped on x86_64-pc-linux-gnu.  Thoughts?

I like it.  Improving re-build time in my dev tree is very much
welcome, and yes,
libbackend build time is a big part of it usually (plus of course cc1
link time).

Richard.

> -- >8 --
>
> Subject: [PATCH] Build libbackend.a as a thin archive if possible
>
> gcc/ChangeLog:
>
>         * configure.ac (thin_archive_support): New variable.  AC_SUBST it.
>         * configure: Regenerate.
>         * Makefile.in (THIN_ARCHIVE_SUPPORT): New variable.
>         (USE_THIN_ARCHIVES): New variable.
>         (libbackend.a): If USE_THIN_ARCHIVES then pass T to ar to build
>         this archive as a thin archive.
> ---
>  gcc/Makefile.in  | 17 +++++++++++++++++
>  gcc/configure    | 20 ++++++++++++++++++--
>  gcc/configure.ac | 13 +++++++++++++
>  3 files changed, 48 insertions(+), 2 deletions(-)
>
> diff --git a/gcc/Makefile.in b/gcc/Makefile.in
> index 0786fa3..15a879b 100644
> --- a/gcc/Makefile.in
> +++ b/gcc/Makefile.in
> @@ -275,6 +275,17 @@ else
>  LLINKER = $(LINKER)
>  endif
>
> +THIN_ARCHIVE_SUPPORT = @thin_archive_support@
> +
> +USE_THIN_ARCHIVES = no
> +ifeq ($(THIN_ARCHIVE_SUPPORT),yes)
> +ifeq ($(AR_FLAGS),rc)
> +ifeq ($(RANLIB_FLAGS),)
> +USE_THIN_ARCHIVES = yes
> +endif
> +endif
> +endif
> +
>  # -------------------------------------------
>  # Programs which operate on the build machine
>  # -------------------------------------------
> @@ -1882,8 +1893,14 @@ compilations: $(BACKEND)
>  # This archive is strictly for the host.
>  libbackend.a: $(OBJS)
>         -rm -rf libbackend.a
> +       @# Build libbackend.a as a thin archive if possible, as doing so
> +       @# significantly reduces build times.
> +ifeq ($(USE_THIN_ARCHIVES),yes)
> +       $(AR) $(AR_FLAGS)T libbackend.a $(OBJS)
> +else
>         $(AR) $(AR_FLAGS) libbackend.a $(OBJS)
>         -$(RANLIB) $(RANLIB_FLAGS) libbackend.a
> +endif
>
>  libcommon-target.a: $(OBJS-libcommon-target)
>         -rm -rf libcommon-target.a
> diff --git a/gcc/configure b/gcc/configure
> index ed44472..81c81b3 100755
> --- a/gcc/configure
> +++ b/gcc/configure
> @@ -679,6 +679,7 @@ zlibinc
>  zlibdir
>  HOST_LIBS
>  enable_default_ssp
> +thin_archive_support
>  libgcc_visibility
>  gcc_cv_readelf
>  gcc_cv_objdump
> @@ -18475,7 +18476,7 @@ else
>    lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
>    lt_status=$lt_dlunknown
>    cat > conftest.$ac_ext <<_LT_EOF
> -#line 18478 "configure"
> +#line 18479 "configure"
>  #include "confdefs.h"
>
>  #if HAVE_DLFCN_H
> @@ -18581,7 +18582,7 @@ else
>    lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
>    lt_status=$lt_dlunknown
>    cat > conftest.$ac_ext <<_LT_EOF
> -#line 18584 "configure"
> +#line 18585 "configure"
>  #include "confdefs.h"
>
>  #if HAVE_DLFCN_H
> @@ -27846,6 +27847,21 @@ $as_echo "#define HAVE_AS_LINE_ZERO 1" >>confdefs.h
>
>  fi
>
> +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking support for thin archives" >&5
> +$as_echo_n "checking support for thin archives... " >&6; }
> +thin_archive_support=no
> +echo 'int main (void) { return 0; }' > conftest.c
> +if ($AR --version | sed 1q | grep "GNU ar" \
> +    && $CC $CFLAGS -c conftest.c \
> +    && $AR rcT conftest.a conftest.o \
> +    && $CC -o conftest conftest.a) >/dev/null 2>&1; then
> +  thin_archive_support=yes
> +fi
> +rm -f conftest.c conftest.o conftest.a conftest
> +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $thin_archive_support" >&5
> +$as_echo "$thin_archive_support" >&6; }
> +
> +
>  { $as_echo "$as_me:${as_lineno-$LINENO}: checking linker PT_GNU_EH_FRAME support" >&5
>  $as_echo_n "checking linker PT_GNU_EH_FRAME support... " >&6; }
>  gcc_cv_ld_eh_frame_hdr=no
> diff --git a/gcc/configure.ac b/gcc/configure.ac
> index 086d0fc..63052ba 100644
> --- a/gcc/configure.ac
> +++ b/gcc/configure.ac
> @@ -4899,6 +4899,19 @@ if test "x$gcc_cv_as_line_zero" = xyes; then
>  [Define if the assembler won't complain about a line such as # 0 "" 2.])
>  fi
>
> +AC_MSG_CHECKING(support for thin archives)
> +thin_archive_support=no
> +echo 'int main (void) { return 0; }' > conftest.c
> +if ($AR --version | sed 1q | grep "GNU ar" \
> +    && $CC $CFLAGS -c conftest.c \
> +    && $AR rcT conftest.a conftest.o \
> +    && $CC -o conftest conftest.a) >/dev/null 2>&1; then
> +  thin_archive_support=yes
> +fi
> +rm -f conftest.c conftest.o conftest.a conftest
> +AC_MSG_RESULT($thin_archive_support)
> +AC_SUBST(thin_archive_support)
> +
>  AC_MSG_CHECKING(linker PT_GNU_EH_FRAME support)
>  gcc_cv_ld_eh_frame_hdr=no
>  if test $in_tree_ld = yes ; then
> --
> 2.9.2.321.g3e7e728
>