Re: libstdc++ libtool lossage

[Mark Mitchell <mark at codesourcery dot com>]

Thanks to Alexandre, David, and Richard's explanations, I think I
understand what is going on here, and what we should do to fix it.

I. Old Situation
================

First, a summary of the situation in the state that it was not too
long ago.  I believe that in this state we had no correctness issues,
but we did have performance issues.

(In what follows, I use "g++" and "libstdc++" generically; I believe
the Java situation was exactly the same as the C++ situtation.)

1. Both "g++ -shared" and "gcc -shared" created a library linked with
   the shared, but not static, version of libgcc.

2. At final link time, "gcc" linked with the static version of libgcc,
   while "g++" linked with the shared version.

3. "-shared-libgcc" implied linking with both the shared and static
   versions of libgcc, but with the shared library specified first, so
   that pieces of the static library were only used if they were
   needed.  (There is some code that is only available in the static
   version of the library.)

4. To build "libstdc++.so", we used "gcc -shared", not "g++ -shared".
   Both commands had identical behavior with respect to libgcc, but
   the latter implicitly links in "libstdc++.so", which doesn't make
   sense since that's what we're building.

II. Background
==============

Here are some important facts that we used to justify this situation.

1. It is not safe, on many systems, to mix shared and static versions
   of libgcc, if exception handling is in use.

2. We assumed that users linking final executables with "gcc" probably
   were not using exception handling; therefore it was safe for them
   to mix the shared and static versions.  Since linking with the
   static version reduces distribution headaches, and since that's what
   GCC 2.95 did, we decided that linking with the static libgcc was
   appropriate in C.

3. However, even for C shared objects, it was not safe to use the
   static libgcc; that would make using the C shared object in a C++
   program unsafe.  Therefore, all C shared objects needed to be linked
   with the shared libgcc.  This situation is unfortunate in that if
   the shared object is eventually used in a C-only program, we've pulled
   in another shared object (libgcc) that we don't need.

III. Binutils Optimization
==========================

In new versions of binutils:

1. It is now possible to do exception handling registration stuff
   without making function calls into libgcc; therefore C shared
   libraries no longer need to be linked against the shared libgcc
   at all.

IV. GCC Takes Advantage
=======================

So, we, wanting to take advantage of this ability, made some changes:

1. We made "gcc -shared" no longer link against either version of libgcc,
   when the new binutils are found by configure.  This is safe for C
   shared objects.

2. However, we still needed to link libstdc++ against the shared libgcc.
   (I don't quite know why, but that's OK, I don't need to know that.)
   But linking with "gcc -shared" above didn't work any more.

3. So, we decided to try "gcc -shared -shared-libgcc".  That wasn't
   something anyone had ever thought of saying before; it used to be
   essentially redundant since "-shared" already linked with the shared
   libgcc.  (Parenthetical: this is part of why having tons of switches
   is hard; you have to define all the n-way interactions between them.)

4. But, "-shared-libgcc" meant (see I.3) that you link with both
   the shared and static versions of the libgcc.  So, now,
   "gcc -shared -shared-libgcc" doesn't mean what "gcc -shared" used
   to mean; the new command is also linking in the static version of
   the library. This is bad (see II.1).

5. So, we decided to change the behavior of "-shared-libgcc" to
   link only the shared libgcc, thinking that the old behavior was
   either wrong, or at least unncessary.  Unfortunately, this doesn't
   work either because of the fact tha some code is only available
   in the static version of the library (see I.3); therefore linking
   final executables that use this functionality with "-shared-libgcc"
   no longer works.

V. Conclusions
==============

These are not facts; they are my opinions.

0. I kind of wish that shared libraries had never been invented.  They
   have ruined my Sunday morning.

1. It was probably inappropriate to be making these changes at the time
   that we were making them.  We were supposed to be fixing bugs, not
   adding optimizations. By arguing that we were fixing a performance
   regression (load time for C shared objects) we could justify these
   changes, but I find that something of a stretch.  This is a great
   example of how allegedly simple changes can have ripple effects
   that slow things down a lot.

2. The change to "gcc -shared" makes sense; if you don't have to link
   against the shared libgcc, there's no reason you should have to do
   so.

3. The semantic change to the meaning of "-shared-libgcc" in the
   final link context did not make sense; the old semantics were
   logical.

4. There are at least two easy fixes:

   a. Make -shared-libgcc behave differently when linking a shared
      library from the creation of a final executable.  (Now that
      I understand the problem, I see that this was Alexandre's
      suggestion. That we arrived at the same solution independently
      is perhaps a good indication that it is a decent idea.)

   b. Introduce "g++ -im-making-libstdc++-right-now" which magically
      links with the shared libgcc, but not libstdc++.so.  This
      solution would allow us to do additional magic at that stage,
      if we needed to do so.

5. If, for some reason, these changes don't work, we can go back to
   the old behavior for 3.1 and address the performance optimization
   in GCC 3.2.

6. Abstractly, one issue is that the shared and static versions of
   libgcc do not define the same sets of symbols.  It would be simpler
   to think about if they had the same symbols, and stuff that's only in
   the static version of libgcc in some other library were somewhere
   else -- but it's not clear that the benefits of making such a change
   outweight the costs, and it's certainly inappropriate right now.

I prefer V.4.a to V.4.b.  Therefore, I'll pre-approve Alexandre's
proposed change.  Alexandre, please also add documentation explaining
what "-shared-libgcc" does in both the shared library and final link
contexts.

Since we have a solution that I am confident will work, I will create
the branch tomorrow, providing that the SPARC problem has been fixed.
(Has it?)  If nobody implements the solution shortly on the branch,
I will revert the changes back to the known working state (see V.5).

Thanks,

-- 
Mark Mitchell                mark@codesourcery.com
CodeSourcery, LLC            http://www.codesourcery.com