Re: stack overflow

[Godmar Back <gback at cs dot utah dot edu>]

 Hi,

In April, we've had a brief discussion on this list about stack 
overflow detection.  I noticed that it's still not implemented
in the recent snapshot I compiled.

Has there been any design or implementation since?  I've grepped
the mailing list archives, but couldn't find anything.

From what I can tell, there's different possible solutions.
Stack overflow checks can be implemented in hardware, by guarding the stack
with read/write protected pages.  However, it is my understanding that 
guard pages cannot be used portably, because handling and recovering 
from a protection fault on the stack is hard in usermode in certain
cases.  Sparc is said to be such a case.  While it may be possible to 
implement via alternative signal stacks, I believe it is fair to say 
that it is hairy.

The alternative is to implement stack overflow checking in software:
at each procedure entry, we check whether the set stack limit is reached
and if so we throw a StackOverflowError.  Some architecture's (ARM) 
procedure calling conventions recommend to set aside a global register (sl) 
to hold the current stack limit;  this allows this check to be implemented 
in two instructions.

Another possible solution is to rely on the specifics of the underlying
threading system.  For instance, a user-level threading system may hold
the current stack limit in a thread-control block that can be accessed
through a global variable.  Or, the current stack limit can be computed
via address arithmetic from the current stack pointer if stacks are
aligned a certain way (Mach's cthreads aligned them at 64K boundaries.)  
Such solutions have the disadvantage of being specific to the underlying 
threading system, not SMP-safe, or both.

Here's one straightforward solution that Tim implemented in Kaffe.
It that has the advantage of being architecture-independent---it
comes at a cost though.  Nevertheless, because of its simplicity,
it may be something quickly doable for gcj.

At every procedure call, an additional parameter is being passed.
This parameter contains the current stacklimit.  On each procedure
entry, the parameter is checked against the current stack pointer.

This scheme has the advantage of being SMP-safe, independent of the
underlying threading system and not relying on hairy hardware tricks.
As such, it may be a good match for whatever systems Cygnus uses
libgcj in.  Of course, there should be an option to turn stack overflow
checking off in cases where it's not needed.

What do people think?  

	- Godmar