Handling of cast operators in the mangling ABI

[Ian Lance Taylor <ian at wasabisystems dot com>]

When writing the new demangler, I discovered an interesting quirk in
the way g++ handles template parameter substitution for templated cast
operators.  I think this case merits some attention in the mangling
ABI, since I, at least, found it to be rather unexpected.

Consider this test case:

template<typename t> class B { public: t i; B(t j) : i(j) { } };
template<template<typename v> class u, typename v> u<v> f() { return u<v>(0); }
B<int> g() { return f<B, int>(); }
template<typename t> class A { template<typename u> operator u(); int i; };
A<float>::operator int() { return this->i; }

Let's look first at the mangled name for f<B, int>:
    _Z1fI1BiET_IT0_Ev
This demangles to
    B<int> f<B, int>()
Pulling the mangled name apart, we get this:
_Z
 1f               (name 'f')
  I               (start of template arguments)
   1B             (name 'B')
   i              (int)
  E               (end of template arguments)
 T_               (return type of 'f': first template parameter ('B'))
  I               (start of template arguments)
   T0_            (second template parameter ('int'))
  E               (end of template arguments)
 v                (void--i.e., function 'f' has no arguments)

This mangled name first describes f<B, int>, and then describes the
return type of the function.  When describing the return type, the
template parameter references refer to the template parameters used
for f<B, int>.  This is straightforward, and follows the ABI, which
says ``When function and member function template instantiations
reference the template parameters in their parameter/result types, the
template parameter number is encoded, with the sequence T_, T0_, ...''
(to be a gramatical pedant, that should be etc., not an ellipsis, but
that is not important now).  Note in particular that in the sequence
T_IT0_E the template parameter reference T_ refers to the first
parameter of the outer template (i.e., B in f<B, int>) not to its own
first parameter (i.e., T0_, which refers to int in f<B, int>).

Now let's turn to the mangled name for A<float>::operator int():
    _ZN1AIfEcvT_IiEEv
This demangles to
    A<float>::operator int<int>()
Again pulling the mangled name apart, we get this:
_Z
 N                (nested name)
  1A              (name 'A')
   I              (start of template arguments)
    f             (float)
   E              (end of template arguments)
  cv              (cast operator)
   T_             (type of cast--first template parameter (???))
   I              (start of template arguments)
    i             (int)
   E              (end of template arguments)
 E                (end of nested name)
 v                (void--i.e., function has no arguments)

Now, what does the template parameter T_ refer to?  From the source
code, it must refer to the type int.  However, there is only int
available in the mangled name, namely the first parameter of the
subsequent template parameters.  In other words, in this case, when we
see T_IiE, the template parameter reference T_ refers to its own first
parameter, in direct contradiction to the previous case.

The difference is apparently that the cast operator is handled
specially with respect to template parameter substitution.  My best
guess is that when the cast operator is itself a template, as
indicated by the presence of template parameters, any template
parameter used in the name of the cast operator must be resolved
against the list of template arguments used in the definition of the
cast operator.  Note in particular that this is a forward reference in
the mangled string.

I think this needs to be spelled out more clearly in the mangling ABI.
It's true that the ABI mentions the use of template parameters in
parameter/result types, and it's true that the return type of operator
A<A>() is in fact A.  But I think that the fact that a forward
reference is required here, and the fact that it only applies to the
cast operator, merits a special mention in the ABI document itself.

I recommend adding sentences along the lines of ``The instantiation of
a templated conversion operator may use a template parameter number in
the <type> which appears in the <operator-name>.  This parameter
number refers to the template arguments for the templated conversion
operator itself.  Note that this is a forward reference in the mangled
name, as the template arguments will appear after the
<operator-name>.''

Thanks.

Ian