Segfault JvCreateJavaVM when doing dlopen, dlclose, dlopen

[Boehm, Hans]

> From:  Andrew Haley
> 
> Matthijs van de Water wrote:
> 
> > I'm having a problem with a Java .so (compiled with gcj-4.3 
> for ARM) 
> > that I use in a C++ program as a dynamically loadable plugin.
> > The plugin works fine as long as I keep it loaded into 
> memory. Because 
> > I'm working on an embedded system, I'd like to be able to 
> unload the 
> > plugin when it's not in use to save memory.
> > 
> > I'm trying the following:
> > 
> > 1. dlopen() the .so, we call an init function in this .so 
> plugin which 
> > uses JvCreateJaveVM() and JvAttatchCurrentThread() 2. Call some 
> > functions in the .so plugin. These functions use Java objects.
> > 3. Unref all Java objects and call unload function in the .so that 
> > calls JvDetachCurrentThread() 4. dlclose() the .so 5. 
> dlopen() the .so 
> > again (as step 1) 6. The code crashes with a SEGV when making calls 
> > into Java code (as step 2)
> > 
> > I've not been able to properly pinpoint the crash yet, but 
> it seems to 
> > crash outside my own code (in libgcj somewhere).
> > Before I spend hours of debugging, could one of you at 
> least tell me 
> > if what I'm trying to do should be working?
> >
> > The one 'strange' thing I'm doing is that I've statically 
> compiled a 
> > PIC version of libgcj.a into the .so plugin to avoid the huge 
> > libgcj.so on my embedded system. This has been working without 
> > problems so-far, however.
> 
> You're in unknown territory here.  What you're trying to do 
> is pretty hairy, and I have no way to know what the problem 
> you're experiencing is without debugging.
> 
> It would help if you could produce a test case that 
> duplicates your problem.
> 
> Andrew.
> 
I'd immediately worry about several possible problems:

- The runtime presumably creates additional threads that are at least in danger of touching memory that will go away with dlclose.  The thread performing finalization immediately comes to mind.  If you enable parallel GC, the other GC threads will have the same issue.

- The garbage collector is, in general, not prepared to deal with dlclose().  But I can't immediately think of anything that should go wrong in your scenario, so long as you make sure that no pieces of the collector are still running when you unload the collector.  If you're performing a dlclose() but keeping the collector running, there's a problem that a garbage collection may be in progress and tracing static data associated with the library you're about to close.  But if you're unloading the collector itself, and parallel GC is disabled, I'm not sure there should be an issue.

Hans