solving c++ Linkage/binary compatibility problems

[Uwe Pachler <uwe dot pachler at jk dot uni-linz dot ac dot at>]

Hi there!

There is a problem on (as far as I know) all platforms with linking
shared libraries that contain c++ classes. Probably all you know the
problem anyway, but since I don't know a name for it, I'll just describe
it here:
You write a shared library 'libfunny.so', containg a class C that looks
like this:

class C	{
public:
  virtual void m1(int){}
  virtual void m2(int){}
};

Say the library binary you created has version 1.0. You create an
executable that dynamically links this library.

Now you add a method 'additional()' to class C, so that it looks like
this:

class C {
public:
  virtual void m1(int){}
  virtual void additional()
  virtual void m2(int){}
};

And now you rebuild the library in this new version, let's call it 1.1.
The problem is that the old executable, built for version 1.0, can't
link the new library version. 
Another problem is object size. If you create a C instance on the stack
with version 1.0, and then, in version 1.2, a private member is added to
class C, the space allocated on the stack isn't correct any more in you
executable (since when the executable was compiled with version 1.0, the
compiler didn't know anything of a member variable in class C; it was
introduced later).

I know that these problems are 'solved' using a copy of the library for
each version (where ldconfig creates symlinks appropriately), but is
this really the best solution?

Wouldn't it be better to introduce a better runtime linkage mechanism
that resolves method names directly, rather than using indices into a
virtual table that is created upon compile time?

The mechinism would have following features:
(when I say link time I mean linkage to a dynamic library)
* Resolve all method calls on link time.
* Calculate Object sizes (for objects created on the stack, i.e as local
variable) on link time.
* Build vtable on link time (for classes defined in the application that
are derived from classes in the library)
Probably there are a lot more, that I didn't consider here...

Introducing this mechanism would mean to modify both gcc and ld, and
probably to extend the ELF format as well (I don't know much about how
the ELF format works, so basically haven't got a clue ;), but It would
be worth it. OO libraries like Qt (http://www.trolltech.com) or Iris
Performer (http://www.sgi.com/software/performer/) suffer from problems
like this.

In the Qt Technical FAQ
(http://www.trolltech.com/developer/faq/tech.html), Question 8, it says:

"You frequently say that you cannot add this or that feature because it
would break binary compatibility. What does this mean, really? 
--
  Binary compatibility means that you can safely distribute your Qt
programs dynamically linked to the Qt library. If the users of your
program have a newer version of the Qt dynamic library installed (or
later upgrade to one), your program will still work. This can save much
time, network, disk, and memory resources and adminstration work, for
both you and the users of your Qt-based programs.
  Technically, this means, roughly ordered from least to most obvious:
* We cannot add reimplementations of virtual functions, unless it it
safe for older binaries to call the original implemenation. This is
because the compiler evaluates SuperClass::virtualFunction() calls at
compile-time, not at link-time. 
* We cannot add or remove virtual member functions. This would change
the size and layout of the vtbl of every subclass. 
* We cannot change the type of any data members or move any data members
that can be accessed via inline member functions.
* We cannot change the class hierarchy, except to add new leaves. 
* We cannot add or remove private data members. This would change the
size and layout of every subclass. 
* We cannot remove public or protected member functions unless they are
inline. 
* We cannot make a public or protected member function inline. 
* We cannot change what an inline function does, unless the old version
continues working. 
* We cannot change the access rights (i.e. public, protected or private)
of a member function. Some compilers mangle the access rights into the
function name."

Most of the problems addressed here would be solved by a mechanism I
outlined above. 
Now my questions are:
1) Is it possible to create a mechanism like this with C++ (I'd say it
is, but I'm not an expert)
2) If it is possible, why hasn't it been implemented yet? Or are there
compilers/runtime systems that implement something like this already for
C++?

Cheers,

Uwe

-- 

   Uwe Pachler

   VR/Graphics Developer

   ABATEC GMBH
   c/o FIM, University of Linz
   TNF Tower, Rm 665
   Altenbergerstr. 69
   4040 Linz
   
   AUSTRIA
   
   phone: +43 732 2468 8433