Looking for ideas to fix X server crash running GCJ-compiled program

[Scott Gilbertson]

Hans> 1) You should be able to set GC_free_space_divisor to a large number
instead of fixing
Hans>  the heap size.

That's an interesting approach.  I tried 10, which results in really
frequent collection under heavy "garbage load", and a maximum heap size
under six megs.  Unfortunately, each collection takes 200-300ms, which is
kind of annoying.  For snappy response, I'd think incremental GC was the way
to go, but when I tried that the application didn't work at all. I also
found that this setting increased the startup time of my application, but I
suspect I could run with one set of GC settings during startup (using
System.gc() to keep the heap reasonable) and another once startup is
complete.

Hans> If you are running in a workstation environment, this is likely to be
more robust.

It's an embedded system with 128M RAM but no swap.

Hans> 2) Setting GC_max_retries to essentially infinite will essentially
force the collector into an infinite
Hans> GC loop when it runs out of memory.  That's usually not desirable.

The only alternative (using a max heap limit), as far as I can tell,  is to
let the application crash if a few tries doesn't get the free space high
enough.  That's not desirable either. I tried a small number (5, I think)
and it wasn't enough -- the app crashed on "out of memory".

Hans> Does the behavior change if the server isn't local?

Nope.  Same symptoms, including the gdb backtrace details.

Hans> Xscope doesn't shed any light?

As far as I can tell, you can't get Xscope for linux, but I found xmon,
which works well, but hasn't shed any light on the problem.  The problem is,
I think, that there's a backlog of stuff for xmon to print at the time of
the server crash.  You therefore don't get to see the last few messages
before the crash.  The last CopyGC in my trace looks fine (i.e., the "src"
and "dst" GCs have valid IDs and haven't been freed yet at the time of the
copy).

The last few commands in the trace are:
    CreateGC
    CopyGC
    SetClipRectangles
    SetClipRectangles

That's just like the thousands of iterations earlier in the trace, which
then go on to (typical example):
    ChangeGC (foreground & font)
    PolyText16
    FreeGC

I'm thinking of writing an AWK script to check the CreateGC/FreeGC ordering.