C++ demangler ABI vs. g++

[Ian Lance Taylor <ian at wasabisystems dot com>]

Since I just dug through the C++ V3 mangling ABI in spec and in usage,
I thought I should report a discrepancy I found.  I found it because I
implemented the demangler based on my reading of the ABI, and then
found that there were some symbols generated by g++ which the
demangler did not handle correctly.

I can file a PR on this if that would be appropriate.  Basically,
though, I think the mangling ABI should be changed.

I used the mangling ABI found here:
    http://www.codesourcery.com/cxx-abi/abi.html#mangling
(I don't know if this is in the gcc repository anywhere, but I think
it would be appropriate.)

The mangling ABI says that whenever the grammar uses <substitution>,
that is a substitution candidate (with some restrictions which are not
relevant here).  The grammar includes these productions:
    <type> ::= <substitution>
    <pointer-to-member-type> ::= M <type> <type>
This clearly suggests that the second <type> in
<pointer-to-member-type> is a substitution candidate.

However, g++ does not work that way.  Only the base type of the second
<type> is a substitution candidate.  If the second <type> is
CV-qualified, the fully qualified type is not a substitution
candidate.

The ABI specifies that any CV-qualifiers on the second <type> are
qualifiers for the member function which is being pointed to.  So the
second <type> is slightly unusual.  But the ABI does not say that
those CV-qualifiers are not themselves a substitution candidate as is
indicated by the grammar.

In the code in cp/mangle.c, see write_function_type().  For a
pointer-to-member-function, it writes out the CV qualifiers for the
first parameter to the function type, and it passes the function type
to write_bare_function_type(), but it never calls add_substitution()
for the qualified type.  This is understandable, since it does not
have the correct type to pass to add_substitution().  But to me it
looks like a violation of the mangling ABI.

Here is a test case:

class G { int g; }; class H { int h; };
template<class t> class what { int w1; };
template<class t> class what2 { int w2; };
void r(int (G::*)(), int (G::*)() const, G, int (H::*)(), int (G::*)(),
       what<G const>, what2<G const>, int (G::*)() const) {}

The mangled name of the function r is
    _Z1rM1GFivEMS_KFivES_M1HFivES1_4whatIKS_E5what2IS8_ES3_
which demangles into
    r(int (G::*)(), int (G::*)() const, G, int (H::*)(), int (G::*)(), what<G const>, what2<G const>, int (G::*)() const)

Breaking this down, we get (substitution candidates are labelled with
SUB and the substitution index):

_Z                   (prefix)
 1r                  (name 'r')
 M                   (pointer to member)
  1G                 (name 'G') (SUB 0)
  F                  (function)
   i                 (int--return type)
   v                 (void--no parameters)
  E                  (end function) (fn is SUB 1) (ptr-to-mem is SUB 2)
 M                   (pointer to member)
  S_                 (substitution 0 == name 'G')
  K                  (const)
   FivE              (function as above) (fn is SUB 3) (p-to-m is SUB 4) ***
 S_                  (substitution 0 == name 'G')
 M1HFivE             (pointer to member as above)
                     ('H' is SUB 5) (fn is SUB 6) (p-to-m is SUB 7)
 S1_                 (substitution 2 == first ptr-to-mem above)
 4what               (name 'what') (SUB 8)
  I                  (template arguments follow)
   K                 (const)
    S_               (substitution 0 == name 'G') (SUB 9)
  E                  (end template arguments)
 5what2              (name 'what2') (SUB 10)
  I                  (template arguments follow)
   S8_               (substitution 9 == 'G const')
  E                  (end template arguments)
 S3_                 (substitution 4 == second ptr-to-mem above)

So, if you followed all that, look closely at the line with ***.
There you see that the function FivE is a substitution candidate, and
the full pointer to member MS_KFivE is a substitution candidate.
However, by the reading of the grammar, the qualified type KFivE
should also be a substitution candidate.  However, it is not.  If it
were, the rest of the name would be mangled incorrectly, because the
references to substitution 4 and 9 would refer to the incorrect
types.

I think the simplest fix is to change the ABI to say
    <pointer-to-member-type> ::= M <type> <CV-qualifers> <type>
That would clarify that the <CV-qualifiers> do not form a substitution
candidate with the second <type>.

Ian