solving c++ Linkage/binary compatibility problems

[Uwe Pachler]

> 
> The short answer is `speed and efficiency' or `abi stability', pick
> any one.  In C++, we have chosen the first, you seem to want the
> second.  It is hard to just change C++, as then it would be much
> slower, and quite a lot of people would probably not want this.

Well, this is not true. The mechanism I mean doesn't affect run time
caracteristics of the created executable at all. It is only active at
link time (when the loader links the shared library to the executable).

> In C++, one can get all the abi stability one wants, for a certain
> cost through the use of specific coding conventions.  For example, if
> one would write:
> 
> object.h:
> struct A {
>       int i;
> } a;
> int foo(A &a);
> 
> main program:
> #include <object.h>
> 
> int main() {
>     A a1;
>     print(a.i);
>     foo(a1);
> }
> 
> one can instead write:
> 
> object.h:
> #include <Integer.h>
> struct A {
> private:
>       Integer i;
> public:
>       static A* make_A();
>       static A* get_global_a();
>       Integer get_i();
> } a;
> int foo(A &a);
> 
> main program:
> #include <object.h>
> 
> int main() {
>     A *pa1 =  A::make_A();
>     print(A::get_global_a()->get_i());
>     foo(*pa1);
> }
> 
> Notice how the use of ``A'' no longer constrains the actual
> implementation of A to be a certain size.  But, this comes at a
> certain cost, heap allocation instead of stack allocation, extra calls
> into the library, three extra functions in the executable and so on.
> Also, notice that the problem of running out of bits in i goes away.
> Maybe i was used to count dollars, and the economy grew and we needed
> more than 2 billion.

Well. I think I haven't made myself clear enough. First of all I know
this construct you're describing here, and, as you're pointing out, it's
somewhat messy. I also think that creating objects on the stack is in
most situations the most natural thing to do, so I usually prefer it
this way as well.
Looking at the example you gave above, the mechanism I mean (I'll call
it the Extended Runtime Linker, or ERL), would read the size of A from
the library (assuming A is in the shared library libA.so), and
recalculate the stack frames in the main program so that A fits on the
stack. You see, ERL would (of course) only do this once when the program
is linked to the shared library. The with the term 'dynamic linking' I
didn't mean linking at run time, but program load time.
And by the way, the code you presented here doesn't fix the method
insertion problem. There are only really messy ways to get around that,
which usually means to write code in C rather than C++...

> [...]
> 
> Also, take a look at the SGI C++ STL library.  It has few abi
> problems, as there aren't any object files.  The cost is that every
> use of it has to see the full source code of it, all the time.
> However, it does delay creating a binary, until you do your
> application compile.  If all such libraries in a system used that
> method, then the abi would only be set when the final app was
> compiled.  The cost of this technique, would be that there would be no
> sharing of compiled library code between applications, redundant
> compilation of the same code, over and over again and so on.  It is
> this sharing that causes the abi breakage, if you will, so by
> eliminating the sharing, you eliminate the problems.

The STL, as all templates, don't have any ABI I know of. The code for
them is created within the object files they are instanciated in, and
are therefore in a way statically linked to the executable. Where is the
ABI of a code that is created at compile time, and where there is no
binary library for?
To be honest I don't really understand why you take the STL as example
here...

> So, you can get it, but it'll cost you, one way or another.  In the
> longer term, you have a language that is flexible enough so that you
> can get the semantic simplicity in a natual fashion (the first code
> sequence), but will have the semantics more like the second code
> sequence, and the optimizer, as late as run time, will notice you need
> this code to run fast and will on demand get you the short instruction
> sequences of the first code sequence.  In time, just now right now.

But that's the point! The ERL _would_ allow all you're describing above!
And it would work with the first code sequence, at _no_ runtime costs,
since everything is done at program load time. That's why we'd need a
modified version of g++ _and_ ld.

Maybe I should describe in a little more detail, what ERL would do:

g++ creates an executable, with certain symbolic information. I assume
the exectuable uses libA.so, containing class A. Here's the code

//executable.cpp:
int main() {
  A a;
  a.m1();
  a.m2();
}
 
//libA.H:
struct A{
  void m1();
  void m2();
private:
  member_var;
}; 

For the exectutable, g++ would create the usual symbol info, plus:
* a table of locations in the code where objects are created on the
stack and referenced
* a table of locations where stack frames are created
* a table of locations where methods are called. Usually, the compiler
created things like _vtable[method_index](...), here the compiler will
keep the method_index blank (it will be filled in later)
* a table of all methods required for subsequent linkage (very similar
to what's being created at the moment)
* (and some other stuff, but I'll keep that for a later discussion ;)

For libA.so, g++ would also create some additional info:
* The size of A
* The exported methods
* (and again some more meta-info)

What ld (ERL enhanced ;) does at load time is that it loads the image of
the executable first, does some relocation stuff, loads the library
libA.so and:
* recalculates the stack frame sizes and references to instances of A on
the stack for the executable (to get around the size problem)
* fills in the method indices (offsets) into the _vtable references in
method calls(this is to make old executables immune against method
insertions in newer versions of class A)

At the very end, the whole memory image of the executable will look like
the old image we got with the normal linker, which means we won't have
any runtime penalty...

> And some time after that, you'll have library code that will rewrite
> the parts of your application that used an older version of it, so
> that the use of it is more correct, or better, or faster, or more
> maintainable.  You can start to see this feature in things like the
> warnings GNU ld will spit out for gets.  The library tries to get the
> person that does the link to rewrite the application that uses gets,
> to not use it.

I'm looking forward a further discussion about that :)

Cheers, 
Uwe

-- 

   Uwe Pachler

   VR/Graphics Developer

   ABATEC GMBH
   c/o FIM, University of Linz
   TNF Tower, Rm 665
   Altenbergerstr. 69
   4040 Linz
   
   AUSTRIA
   
   phone: +43 732 2468 8433