GCC Bugzilla Bug 24660

This is an experiment in versioning the vague parts of libstdc++. It is
apparent that some kind of control over weak symbols is necessary for accurate
versioning of C++ constructs WRT shared libraries.

To that end, the obvious point of influence is the namespace name itself, since
it is part of the mangled symbol name. 

When trying to version with namespace names, there are a couple of problems.
See Sutter, N1344 for more details and a general problem statement. There
appear to be two general solutions.

One, relax specialization rules WRT declared namespace. Motivations for the
current rules appear somewhat vague, but are best described here:

http://gcc.gnu.org/ml/libstdc++/2005-09/msg00081.html

Two, invent a mechanism to allow lookup and specialization. This was done for
g++ as part of the debug mode work, and is called namespace association.

This patch uses approach two. This was anticipated and discussed as part of the
Kona discussion, and was also subject of a paper at last year's GCC summit.

There are a couple of outstanding issues with this patch.

1) --disable-symvers for the time being

2) ext, "C" headers, and libsupc++ bits need to be thought through

3) longstanding issues with things like std::swap, see other PR's

4) new issues with mainline. See attached for problem statement.

This bug report is really just a convenience for me.

Created an attachment (id=10130) [edit]
libstdc++ patch

Created an attachment (id=10131) [edit]
mainline chokes on this

Created an attachment (id=10132) [edit]
mainline chokes on this


Will attach the compiler output as a separate piece. It looks recent, as
gcc-3.4.x, gcc-4.0.x are ok.

Created an attachment (id=10133) [edit]
mainline output

Isn't in a way strong using what you want here?

namespace association is the correct name for "strong using."

Indeed, you'll find that this is what I am using. 

-benjamin

Here's a reduced testcase:

namespace N { }
namespace std
{
  using namespace N __attribute__ ((strong));
}
namespace N
{
  using namespace std;
  template<typename T>
  struct A
  {
    void f();
  };
}
namespace std
{
  template<>
  struct A<int>
  {
    void f() { }
  };
}

This code, like the testcase for c++/16021, works fine if the implementation
namespace is nested within std.  I'm thinking of enforcing that requirement in
parse_using_directive; that would allow us to avoid adding more special lookup
rules.  Do you see any problem with that requirement?

The nesting also means you don't need the reciprocal using-directive which you
added to avoid the problem from 16021.

Subject: Re:  versioning weak symbols in libstdc++

"jason at gcc dot gnu dot org" <gcc-bugzilla@gcc.gnu.org> writes:

| This code, like the testcase for c++/16021, works fine if the implementation
| namespace is nested within std.  I'm thinking of enforcing that
| requirement in parse_using_directive; that would allow us to avoid
| adding more special lookup rules.  Do you see any problem with that
| requirement? 

I'm for minimal invasion in the name lookup rules; so if that takes
care of issues raised by Benjamin, I'm all for it.

-- Gaby

Created an attachment (id=10271) [edit]
Kona paper, as revised but not in post-Kona mailing

Created an attachment (id=10272) [edit]
Outline for Kona presentation


I cannot believe I found this stuff.

Here you go Jason. Actually, it looks like I have a complete archive of most
parts of this discussion, the related material, code samples, issues, etc. If
you would like, I will tar it up and put it in this pr. Or, if you are
interested, I can send it to you (or others) via private email. 

I should have just put this stuff in post-Kona: sorry. I need to get better
about this kind of thing.

Anyway. Jason, I think making the namespace nested is fine. Interestingly, I
reviewed these discussions, and it looks like at the time of the debug mode
design discussion, we were wondering about the pros and cons of nested vs.
separate namespaces.

Looks like now we know more about these tradeoffs.

The only drawback that I can see is trying to reduce the size penalty for
mangled names. This is going to be an issue for some people, and we might as
well admit it from the start. 

I'd thought of nested solutions, including the obvious ones:

namespace _6 {
namespace std {

namespace std {
namespace _6 {

but the problem is then that all the specialized compression for narrow std::
types (basic_string, basic_istream, etc) will fall down, and all your names go
through two namespace manglings. I'm not convinced this is such a bad thing,
actually: having non-compressed std:: would mean post-processing of library
binaries with other tools will become much more regular, without substitutions. 

Of course, 

namespace _6 {

also removes the specialized namespace std:: compression, but the fully
qualified name is still within reasonable limits of the current status quo.
This was judged to be the least obese of all options, in terms of symbol size
explosion. (ie _ZNS vs. _ZN2_6).

I must tell you that I would rather have this feature (and a resolution to the
weak symbol vs. shared object problem), and loose compression, than have either
the benefits of compression or a shorter fully qualified name. If I had a magic
wand, of course, I'd pick compression on generic template type name, not
instance name, so that wchar_t and char types would also be compressed 
(ie compress basic_string, not basic_string<char>), plus namespace association.
That might be a way to win back some of the loss if we switch to something like
std::_6 for the namespace.

Anyway.

Subject: Re:  versioning weak symbols in libstdc++

I think nesting _6 within std makes sense in the abstract as well, as it 
is properly part of the standard library space, not a separate entity.

The debug mode headers will need to be modified to work with this new 
requirement by moving __gnu_debug_def into std as well.  Do you plan to 
version the debug mode containers, too?

I don't think there's much compression to be done once you start 
inserting arbitrary namespaces.  Just compressing the NTBS 
"basic_string" doesn't get you much to compensate for the added complexity.

You might as well attach the tarball to the PR for posterity.  I'm 
particularly interested in the discussions you mention about the 
relative merits of nesting or not.  I note that the documentation of the 
feature uses a nested implementation namespace, so that seems to be what 
I had in mind...

Jason

> I think nesting _6 within std makes sense in the abstract as well, as it 
> is properly part of the standard library space, not a separate entity.

Sounds sane to me.

> The debug mode headers will need to be modified to work with this new 
> requirement by moving __gnu_debug_def into std as well.  

Yep. Current debug mode will have to be adjusted. If you put your patch in for
the compiler bits to this bug report I can do the debug mode adjustments.

> Do you plan to version the debug mode containers, too?

Don't know. I've not thought that much about it, simply because we don't worry
that much about debug mode and versioning, since it is not on by default. I
think there are only 4 or so exports for debug mode. 

I suppose it should be.

I am thinking all things with templates should be versioned. That would mean
debug, tr1, ext headers, as well as std, and some of the C++ versions of "C"
headers (cmath, for instance, so we'd just do all the "C" headers as well.) 

> I don't think there's much compression to be done once you start 
> inserting arbitrary namespaces.  Just compressing the NTBS 
> "basic_string" doesn't get you much to compensate for the added complexity.

Sure, sure. I think we are on the same page here, actually. 

Created an attachment (id=10275) [edit]
Patch to require nesting

I've attached my compiler patch to enforce this rule.  It can't go in until the
debug mode code is fixed.

Two extra fails with both versioning and debug associations active.

FAIL: 20_util/memory/16505.cc (test for excess errors)
FAIL: 25_algorithms/search_n/11400.cc (test for excess errors)

For the first:

#include <memory>

// libstdc++/16505

struct S { };

template
  void
  std::uninitialized_fill_n<S*, int, S>(S*, int, const S&);

gives:
/mnt/hd/src/gcc.versioned/libstdc++-v3/testsuite/20_util/memory/16505.cc:31:
error: 'void std::uninitialized_fill_n(S*, int, const S&)' should have been
declared inside 'std'


and for the second:
#include <algorithm>
#include <functional>

struct Integral { operator int() const; };

namespace std
{
  template int* search_n (int*, int*, Integral, const int&);
  template int* search_n (int*, int*, Integral, const int&, greater<int>);
}


gives:
/mnt/hd/src/gcc.versioned/libstdc++-v3/testsuite/25_algorithms/search_n/11400.cc:30:
error: 'search_n' is not a template function

Still testing out the other configs. I'll put reduced test cases in bugzilla.

Created an attachment (id=10475) [edit]
test case for first fail

Created an attachment (id=10476) [edit]
test case for second fail

Created an attachment (id=10483) [edit]
libstdc++ patch v5


Config not quite 100% correct, but good enough.

This moves all debug mode stuff to nested within std. In addition, there is a
version mode too, which versions all of namespace std. 

To get to the versioned mode, configure with
--enable-symvers=gnu-versioned-namespaces

In addition, the generated c++config.h has to uncomment:

// #define _GLIBCXX_NAMESPACE_ASSOCIATION_VERSION 1

Without versioned mode, there are no new fails. I'm still testing with the
debug mode turned on.

With versioned mode, there are two new fails. 

Still, not so bad, not so bad. 


Kind of a mouthful at the moment.

This arrangement of debug mode does indeed seem to fix the longstanding swap
issue. 

ie, -D_GLIBCXX_DEBUG runs are == normal runs.

(In reply to comment #20)
> This arrangement of debug mode does indeed seem to fix the longstanding swap
> issue. 
> 
> ie, -D_GLIBCXX_DEBUG runs are == normal runs.

Yeah! (about the rest of the work too ;)

This patch is complete with one exception, libsupc++. It looks like
--disable-hosted-libstdcxx is broken right now, so I have to fix that and then
make it work with this scheme.

http://people.redhat.com/bkoz/libstdc++-v3.versioned-7.tar.bz2

By default, the behavior of this patch is compatible with
gcc-3.4/gcc-4.0/gcc-4.1. I remapped the new, nested symbols to the old exports. 

In addition, with --enable-symvers=gnu-versioned-namespaces, the following
namespaces are versioned: std, __gnu_cxx, std::tr1. Ie, these namespaces are
all  associated with nested namespaces of the type _6.

There are a lot of renames in this, so I just put up all the sources.

When doing the last bit, I ran into some other issues, where some of the
libsupc++ bits (typeinfo, unexpected_handler, uncaught_exception, nothrow etc)
are internally defined within std. 
(? Or seem to be.)

This hardcoding should be adjusted.

Here are three pre-processed examples.

Created an attachment (id=10524) [edit]
libsupc++ issue 1

Created an attachment (id=10525) [edit]
libsupc++ issue 2

Created an attachment (id=10526) [edit]
libsupc++ issue 3

Created an attachment (id=10567) [edit]
files from kona meeting

Created an attachment (id=10568) [edit]
mail surrounding namespace association development

Subject: Bug 24660

Author: jason
Date: Thu Jan 12 20:10:47 2006
New Revision: 109647

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=109647
Log:
        PR libstdc++/24660
        * pt.c (check_explicit_specialization): Handle namespace
        association.
        * name-lookup.c (set_decl_namespace): Likewise.

Modified:
    trunk/gcc/cp/ChangeLog
    trunk/gcc/cp/name-lookup.c
    trunk/gcc/cp/pt.c
    trunk/gcc/doc/extend.texi

Confirmed. Excellent, thanks. This is great, no regressions known with this on.

This experiment is deemed sucessful.