RE: System.gc() to close file descriptors (was RE: status of gcj's bo ehm collector?)

["Boehm, Hans" <hans_boehm at hp dot com>]

Very interesting observation.  I hadn't thought about that problem.  It
seems to me the right solution involves:

1) Blaming the Java programmer if the finalizer that releases the file
descriptor needs a lock that might be held while trying to open a file.  I
think there's no way to solve that problem.  That will cause the open to
fail after a timeout.  You should minimize the locks you hold while
allocating resources.

2) Allowing multiple finalizer threads to bypass blocking by unrelated
finalizers.  GC_invoke_finalizers() is already written so that it should be
safe to call from multiple threads, and only one will attempt each
finalizer.  The wait in System.runFinalization needs a timeout.  If the
timeout expires then it should:

a) Return if the finalization queue is empty, i.e.
GC_should_invoke_finalizers() returns 0.

b) Check if any finalizations have completed since we started or since the
last timeout.  (I think it's easy to maintain a global counter.
GC_invoke_finalizers() returns the number of finalizers it ran.)  If so,
just wait again.

c) If no finalizers were completed, fork a new finalizer thread and wait
again.

The finalizer threads should notify all runFinalization() waiters if a
GC_invoke_finalizers() call returns successsfully.  (At that point it should
probably also tell all other finalizer threads to go away if/when they
finish.)

I admit this sounds messy.  But I'm not sure it's really that hard to
implement.  It means that runFinalization may return before all finalizers
have been run.  But it's only supposed to make a "best effort".  I think it
has very minimal performance impact, since it's not a lot of code, and I
would expect it to be very rare that we need to fork even a second
finalization thread.  It means that runFinalization() will return
eventually, though if there are many finalizers and all of them block, it
may take a long time.  (Perhaps we really want a limit on the number of
finalization threads?)

A simpler, though I think less satisfactory, alternative would be to have
runFinalization just return if the initial wait times out.  Arguably, that
still meets the spec, though it may overlook many finalizers.  It's
equivalent to limiting the number of finalization threads to one in the
above scheme. 

Hans

> -----Original Message-----
> From: Andrew Haley [mailto:aph@cambridge.redhat.com]
> Sent: Thursday, December 20, 2001 5:11 AM
> To: tromey@redhat.com; Boehm, Hans; 'Adam Megacz'; java@gcc.gnu.org
> Subject: Re: System.gc() to close file descriptors (was RE: status of
> gcj's bo ehm collector?)
> 
> 
> Andrew Haley writes:
>  > Tom Tromey writes:
>  >  > >>>>> "Hans" == Boehm, Hans <hans_boehm@hp.com> writes:
>  >  > 
>  >  > Hans> I looked at the gcj code a little more closely.
>  >  > Hans> System.runFinalization() also seems to have two 
> other problems:
>  >  > 
>  >  > I think we should get in the habit of reporting all 
> these sorts of
>  >  > things to Gnats; we've dropped too many issues like 
> this in the past.
>  >  > 
>  >  > Andrew, are you still looking at this?
>  > 
>  > Yes, but it's fairly low priority.
>  > 
>  > I was thinking of running the finalizers in the 
> finalization thread,
>  > but making System.runFinalization() wait until it has been 
> done: it's
>  > easy enough to make finalization notify any waiting threads.
>  > 
>  > I'm concerned about deadlock and race conditions, though.  Comments
>  > welcome.
> 
> I looked at it some more.  Hans's suggestion seems good, but it's
> tricky to implement.  It requires waiting until the finalization
> thread has run, but AFAIK there isn't any way to guarantee that the
> thread needing a file handle isn't holding some lock that prevents a
> finalizer from running.  In other words, the cure is worse than the
> disease.
> 
> Also, I have talked to a Linux kernel engineer.  He tells me that the
> hard limit on file handles in Linux has been removed "because of
> Java."  I don't think that this is a reference to gcj in particular,
> so other Java implementations have the same problem.
> 
> Andrew.
>