RE: PR java/1373: recursion stress test causes segmentation fault

["Boehm, Hans" <hans dot boehm at hp dot com>]

I fully agree with just aborting on stack overflow.  It would be really nice
to get that version of the patch in.

I suspect there are more serious issues with throwing StackOverflowError.
The problems aren't restricted to the GC, though that's part of it.
In fact, thinking about this some more, I'm not sure how this can be
safely handled in standard JVMs either.

In general, a StackOverflowError can occur at arbitrary points, while
a global data structure is being updated.  This applies to Java routines,
libc routines, and everything in between.  In the case of a libc routine,
I think there is no guarantee that even critical locks will be released
when we unwind.  For a Java routine, unwinding will usually release the
locks, but can leave data structures in an inconsistent state.

We could conceivably handle the libc and GC issues by always doing a stack
probe ahead of time.  But it seems to me that even for Java code, this
is basically a bad idea for most of the reasons that Thread.stop() is.
In particular, malicious code could essentially force library code to
corrupt its data structures by calling it on a larger and larger stack.
Thus to guarantee security, library code would have to be written to
ensure that it never does anything bad when started in one of these
corrupt states.  That strikes me as a hopeless burden.

This is all much less of an issue if you abort on stack overflow.  Plus
it's a lot easier to implement.

(Another issue with StackOverflowError: Is -fnon-call-exceptions sufficient
to deal with exceptions on stack accesses, e.g. register spills?)

Hans

> -----Original Message-----
> From: java-patches-owner@gcc.gnu.org
> [mailto:java-patches-owner@gcc.gnu.org]On Behalf Of Andrew Haley
> Sent: Wednesday, October 20, 2004 8:56 AM
> To: java-patches@gcc.gnu.org
> Subject: PR java/1373: recursion stress test causes segmentation fault
> 
> 
> Thinking about this some more, the right way to handle this patch is
> to check it in, but such that by default a stack overflow causes a
> message to be printed and then abort.
> 
> At some point in the future when we've hardened the runtime we could
> cause a stack overflow to throw a StackOverflowError.
> 
> Andrew.
>