Re: Idea for export Implemnation

[Kevin Atkinson <kevinatk at home dot com>]

Joe Buck wrote:
> >
> > The basic idea is to have the template code (either parsed or raw) as
> > data blocks in the object files and have the linker compile them as
> > necessary.
> 
> Depending on how the compiler is configured, we would use either the GNU
> linker, or collect.  Any solution would need to work with both options.
> Furthermore we have to have a solution that works OK with libraries.

True.  As I see it there is no way to use nothing but the system linker
with templates.

> 
> > For example:
> [ Example deleted: foo.hh declares templates, foo.cc defines and exports
>   these templates, main.cc uses the templates without including foo.cc ]
> 
> > g++ -c main.cc:  object code for main gets stored in main.o
> >
> > g++ -c foo.cc:  actual c++ code for Foo<T>::average_data() get stored in
> > foo.o
> 
> I'm not crazy about naming something .o if it's not an object file.  It
> would make it harder to use ordinary linkers.
> 
> An extra file could be produced that contains the exported templates in
> some easy-to-process form.  Or I suppose you could do something evil,
> like store template code as string data in the object file.  But if you
> have ordinary .o files, you can link them with an ordinary linker, and
> only try an extra pass if you have undefined symbols.

Why is storing them as strings eval?  It will allow the necessary
template code to automatically get included in a spot where it is
visible.
> 
> > g++ main.o foo.o -o main
> 
> One nice feature of what you suggest is that the user can ignore the
> fact that there is template expansion going on.  This may be tricky
> to achieve in practice though.

Yes that was the general idea.

> 
> > Now things get interesting.  Where should the compiler store the new
> > code needed for Foo<int>::average_data() the most logical place for it
> > would be foo.o.  So now foo.o contains the c++ code for
> > Foo<T>::average_data() and the object code for Foo<int>::average_data().
> 
> I'm afraid that won't do very well when foo.o might be in a library or
> be included in multiple executables, or if foo.o has a mixture of ordinary
> code and other code.  You try to work around this with special tricks for
> read-only libraries, but I'm not happy with them, since I don't think they
> will work reliably for multiple simultaneous users.  I don't think that
> you should try to first generate foo.o, and then modify it.  That doesn't
> work well with "make": I might change foo.cc but still have a newer foo.o,
> if I first compile, then change foo.cc, then link.

Um How?

Initial: foo.cc 01:00 foo.o <nonexistent>
Compile: foo.cc 01:00 foo.o 02:00
Modify:  foo.cc 03:00 foo.o 02:00

Now when you try to like make should see that foo.cc is older then foo.o
and thus
recompile foo.cc before linking.  When ever foo.cc gets recompiled the
object file gets destroyed and recreated.  It is only when the linker
needs a non existent instantiation that foo.o gets written to without
being destroyed first  So, where is the the problem.

Now if foo.o is included in multiple executables than the linker will
need to see if each template instantiation is necessary and only include
the necessary ones.

About libraries I admit that neither off those solutions is great so how
about this one.

Have a separate file the same name but different extensions that holds
all the instantiations and visible by all users on the system.  So when
an instantiation is needed the linker looks for it in this file.  And if
it is not there it creates the object code for the instations.  Locks
the file somehow.  Writs the new information to the file.  And then
unlocks the file.  When every the library file gets changed it is the
libraries responsibility to purge this instantiation file.  The linker
will then recreate any necessary instantiations it needs when it needs
it.

However with shared libraries this creates a problem because there could
be running programs that already relay on those instantiations.  So what
to do?

1) Have the instantiations file be recreated instead of purged when the
library is updated.

2) Have the shared programs keep there own copy of the instantiations it
needs.  It will try to use the code in the shared instantiations file
however if it can't find it it will try to call the compiler to recreate
the instantiations with the new code.  However if that fails it will
simply use its own copy.

The problem with #1 is that the instantiations file could get cluttered
with unnecessary instantiations over time as it can NEVER be purged in
fear of breaking existing programs.

The problem with #2 is that A) Two copies of the object code are needed
but that is only on disk space and B) If the program is unable to create
the new instantiations it will be forced to use an older obsolete code. 
However this isn't so bad as if the code wan't shared in the first place
it be using its own copy of everything anyway.  Beside often when shared
libraries get updated code needs to be relinked anyway.

The one thing I DO not recommend is having one large template
instantiations database.  As this can lead to all sorts of version
problems.  I think that instantiations should be broken up by object
file or libratie file so that when the object file gets destroyed so
does the instantiations.  Thus there is not the problem with the
instantiations becoming outdated.
-- 
Kevin Atkinson
kevinatk@home.com
http://metalab.unc.edu/kevina/