Organization and symbol lookup for shared libraries

[Andrew Haley]

Bryce McKinlay writes:
 > Andrew Haley wrote:
 > 
 > >I've been thinking more about Bryce's comments.
 > >
 > >
 > >Object: A clean and simple way for gcj users to deploy precompiled
 > >Java packages.
 > >
 > >
 > >Background: On the gcj-abi-2-dev-branch is a new scheme that uses a
 > >database to look up precompiled class files using the bytecode array
 > >of a .class file as the key.  This is good if you have the .class
 > >files around, but not if you don't.
 > >  
 > >
 > Just a bit of additional background: obviously the cleanest and simplest 
 > way to deploy compiled applications that depend on shared libraries will 
 > likely continue to be to explicitly link them to their shared libraries, 
 > ie using "-l" at compile time.
 > 
 > What we're talking about here is how to resolve the native-compiled 
 > dependencies of bytecode applications, or perhaps for natively-compiled 
 > applications where for some reason you don't want to explicitly link the 
 > dependencies. ie: this is a replacement for the existing lib-foo-bar.so 
 > name-based class->shared library resolution mechanism.

Right, a way that you can just issue the command "gcj foo.jar
--main=barf" and have it work, as long as the packages that you need
are installed somewhere on the system.

 > >However, users of gcj want a quick way to deploy precompiled libraries
 > >that doesn't require installing .jar or .class files.  They want to
 > >add many packages without slowing down their programs.
 > >
 > >
 > >So, I propose we do this:
 > >
 > >At startup, libgcj will look in each directory in the java.ext.dirs
 > >path to see if there exists a database file.  If there is one, libgcj
 > >will add it to the gnu.gcj.precompiled.db.path.
 > >  
 > I think it may make more sense to use something like a "GCJLIB"
 > system property or "gcj.solib.path" property rather then re-use
 > java.ext.dirs here. We could also define a well-known location such
 > as /usr/lib/gcj to be the default value for this. rpm packages
 > could then install to this location by default and run the dbtool
 > to update the database.

That seems reasonable, but it's a bit more complex than having a
java.ext.dirs path that simply contains .so and .jar.  Can you think
of any circumstances where you would want to add a directory that
contains .jar files to the path but you would *not* want the .so files
in the same directory?

 > There remains the issue of how exactly to deal with multiple versions of 
 > the same classes when they appear in different shared libraries on the 
 > "GCJLIB" path (ie version skew). Unless someone can think of a good way 
 > to solve it, I propose we just ignore it for now, but allow users to 
 > specify a search order via an "SOCLASSPATH" property. The search order 
 > would thus be:
 >
 > 1. anything explicitly linked

I guess this implicitly include libgcj.so.

 > 2. shared libraries specified in "SOCLASSPATH", in the order specified
 > 3. libraries in system "GCJLIB" dirs, in an ad-hoc order
 > 4. ... start looking for bytecode ...

The problem with SOCLASSPATH is that you can't control the relative
search order of .jar and .so files.  That is, there's no way to say.

  foo.so, bar.jar, bar.so

which IMO you are very likely to want to do.  Allowing .so files in
the classpath is IMO the Right Thing here.

 > >To make deployment easy, the directory that contains the database may
 > >contain a makefile, which in is simplest form will be like something
 > >rather like this:
 > >
 > >----------------------------------------------------------------
 > >gcj.so.db: *.so *.so.*
 > >	gcj-dbtool --addlibs gcj.so.db *.so *.so.*
 > >----------------------------------------------------------------
 > >  
 > >
 > 
 > Perhaps this step (running gcj-dbtool) could be avoided altogether: if 
 > the cache is found to be out-of-date at runtime, because someone added 
 > or changed a shared library without re-running dbtool, then we'll need 
 > to open those modified shared libraries at runtime anyway. Since we have 
 > to open them, the runtime could simply update the cache file on-the-fly?

That's true.  I'm not sure it makes a huge difference, but it would
work.  There are potentially some nasty concurrency problems.

 > This wouldn't completely eliminate the need for dbtool, for example it 
 > would still need to be run by an administrator in order to update a 
 > cache file for shared libraries in a system directory, but it would save 
 > the user from having to remember to run dbtool whenever a new library 
 > appears.

Well, it doesn't appear, exactly -- you have to copy it there!  But
you have a point.

Andrew.