Out of memory problem

[Suresh Raman <sugansha at yahoo dot com>]

Hi,

I wrote a simple program to simulate an OutOfMemory Exception I
started getting when I started using gcj (instead of Sun's JDK).
This is a long posting, I apologise for that.

The sample program starts by allocating a 2K by 1K two-dimensional
byte array.

It then loops 10000 times doing the following in each loop:
     - pick a random number between 0 and 2K
     - go to the index specified by the random location, on the
       first dimension of the array
     - replace the contents of that location with a brand new 1K
       array.

So, the program does not use more than 2MB of memory (plus a small
overhead for other objects) at any instant of time.

The source code listing is at the end of this posting.

My system is a Intel Pentium 1.4GHz, 256MB RAM having:

---Mandrake Linux OS---
Linux 2.4.8-26mdk i686 unknown
-----------------------

---Sun's Java---
java version "1.3.1"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.3.1-b24)
Java HotSpot(TM) Client VM (build 1.3.1-b24, mixed mode)
-----------------

---GCJ/GCC build---
gcj (GCC) 3.1
built with the following options
--enable-threads=posix --enable-shared --enable-languages=c++,java
-------------------

When I build and run the program with gcj,
(build command: gcj --main=leakexample leakexample.java)
it runs fine, but it consumes 6508544 Bytes of heap memory, as
reported by the call to Runtime.getRuntime().totalMemory().  This is
not a problem per se.

But, when I limit the maximum heap size of the program to 4MB* and run
the same program again, I get "Warning: Out of Memory!  Returning
NIL!" errors.   It happens nearly everytime during the 768th pass in
the loop.  The total heap memory is almost at 4M and the free heap
memory is at 0, when the error happens.  Given that the program does
not use more than 2MB at any time, I expect it to run without any
problems within 4MB.  This is unfortunately not the case.

* I limited heap size by adding the following line of code in the
libjava/gnu/gcj/runtime/natFirstThread.cc file, before calling
real_main.
_Jv_GCSetMaximumHeapSize(4*1024*1024L);
** I also tried "gij -Dmx=4194304 leakexample.class" - same result

However, with Jun's JDK, the program runs fine, with only 4MB of
total heap.  (javac leakexample.java; java -Xmx4m leakexample)

I tried experimenting by sleeping for 20ms once every 5 times in the 
loop.  This had no effect.

However, calling System.gc() once every 5 passes in the loop fixes 
the problem.   But I don't find this a good solution...because in a 
real program, it would be a very difficult to identify the areas that 
require a forced gc(), and calling it too often would incur 
performance penalties.  Besides, I am not even sure I can solve the 
problem in my actual application by calling System.gc() in various 
places.

My application runs in a embedded environment with memory 
constraints.  So, any help towards solving my problem would be 
appreciated.

thanks,
--Suresh Raman

---------source code, cut below this line-------------
public class leakexample
{
   private byte[][] bigarray;
   private StringBuffer strBuf = new StringBuffer();
   private final String commaString = ", ";

   public leakexample()
   {
      bigarray = new byte[2*1024][];

      System.out.println("initializing ...");
      for (int i=0; i<bigarray.length; ++i)
      {
         bigarray[i] = new byte[1024];
         for (int j=0; j<bigarray[i].length; ++j)
         {
            bigarray[i][j] = 3;
         }
      }
      System.out.println("done");

      System.out.println("Output format is:");
      System.out.println("Attempt#, Location Index, Total Mem, Free
 Mem");

      java.util.Random random = new java.util.Random();
      int randval;
      for (int i=0; i<10000; ++i)
      {
         // get a random number between 0 and bigarray.length
         randval = random.nextInt(bigarray.length);

         // replace the existing 1K array at that random location
         // with a new 1K array, and initialize the new 1K array
         bigarray[randval] = new byte[1024];
         for (int j=0; j<bigarray[randval].length; ++j)
         {
            bigarray[randval][j] = 3;
         }

         // experiment with calling gc ourself and/or sleeping
         // for sometime once every 5 loops.
//       if (i % 5 == 0)
//       {
//          System.gc();
//          System.out.println("forced GC");
//          try{Thread.sleep(20);}catch(InterruptedException ie){}
//          System.out.println("slept");
//       }

         // construct the line to send to stdout
         strBuf.delete(0, strBuf.length());
         strBuf.append(i);
         strBuf.append(commaString);
         strBuf.append(randval);
         strBuf.append(commaString);
         strBuf.append(Runtime.getRuntime().totalMemory());
         strBuf.append(commaString);
         strBuf.append(Runtime.getRuntime().freeMemory());

         // send to stdout
         System.out.println(strBuf);
      }
   }

   public static void main(String args[])
   {
      new leakexample();
   }
}
-----------end source code, cut above this line------


__________________________________________________
Do You Yahoo!?
Yahoo! - Official partner of 2002 FIFA World Cup
http://fifaworldcup.yahoo.com