Re: Finding memory leaks

[Thomas Aeby <aeby at graeff dot com>]

On Sat, 2004-02-28 at 18:40, Rutger Ovidius wrote:
> I'd be very interested to hear what you find out, if anything.
> 
> Does your app collect properly with sun's java?

I haven't tried this out since I wiped out the last memory leaks that
were truly found in my app ... But actually it would be a good idea
to check this ...

In the meantime I added my own simple mark algorithm to my heap
analyzing Perl script. The results after a few days of debugging are
not very clear, however. What I observe is:

  1) Statistics: After 4 few days of running the RSS of my processes
     is 22MByte - /proc/*/maps tells me that the data segment is 19MB.
     GC_dump() lists 10MByte memory in the free list and 4.5MByte
     blocks in use. When going through the blocks and counting each
     object's size I end up with 1.3MBytes memory used. 700MBytes of
     this is - after my mark algorithm - not in use but still marked
     by gcj's garbage collector. A very small amount of memory (haven't
     even bothered counting) is blacklisted.

  2) Many of the objects that are marked by both my mark algorithm and
     gcj's garbage collector are pointed to by some nested data
     structure that do not constitute Java  objects - I haven't found
     out yet what this is is. Maybe some internal list used for
     incremental garbage collection?

  3) No idea why the garbage collector does not collect the objects that
     are not marked by my algorithm. Mostly they seem to be objects
     that were only allocated locally in a method or even loop. I'm
     doing the inspection after hours of idling - I don't think the
     garbage collector just haven't had enough time for collecting ...

     Maybe the garbage collector decides to stop collecting because
     with actually 70% free blocks it's not really necessary to spend
     time on collection - however, this makes things extremely difficult
     to debug and might be a problem for my app that does just nothing
     for long time periods (hours) and then gets very active (regarding
     memory allocation) for a few seconds or minutes and goes idling
     again, after. Maybe the garbage collector just isn't collecting
     fast enough during "active" periods and refuses to collect
     when there were time to do it?

  4) After having a closer look at some un-collected objects and
     comparing them to the ones that were collected I started

	- declaring classes defined locally within other classes
          "static" wherever possible in order to avoid objects being
	  linked with their "parents"

	- use local variables rather than instance variables wherever
 	  this is possible.

	- keep the lifetime of objects with pointers to other objects
	  as short as ever possible.

      This seems to reduce the number of objects that actually stay
      uncollected. But this is rather a feeling than a proven fact ... I
      see that the same code may allocate objects that are "immediately"
      collected while from time to time some objects seem to get "lost"
      somehow and are not collected ... sometimes they are collected
      when I have a look at the heap hours later.

      For instance I used to put some Thread objects into a Vector
      and regularly (all 30seconds) I had a loop going through this
      Vector, wiping out Threads that weren't active any more. One
      of this Thread derived classes had a char[16384] allocated in an
      instance variable. I saw that randomly but rather often these
      char[] were not collected even after the object was removed from
      my Vector. After allocating the char[] within the run() method
      rather than the instance variable there are still some rare
      cases where a stray object is left marked on the heap but it's
      not a big problem any more. In the meantime I gave up my Vector,
      too ...

   5) The total heap size seems to be slowly (well, not constantly, 
      but in increasingly large chunks) increasing though the amount
      of memory in use currently stays more or less the same (started
      at about 900kBytes, after 4 days it's now 1.3Megs). Of course,
      this is the "idle" state of my app. From time to time it's doing
      some work and temporarily uses 3-4Mbytes - mostly in many very
      small objects, but also allocating some 4-32kByte buffers. No
      idea, why the heap keeps growing - maybe due to fragmentation?
      Maybe due to the garbage collector not collecting fast enough?
      I don't know. Unfortunately, my way of debugging is not "real
      time" capable - I just can do "heap shots" at arbitrary moments.

      Sooner or later the application just crashes - probably due to
      memory problems.

   6) Unfortunatley I haven't found any way to switch GC debugging on
      with gcj. I assume the backtrace function would enlighten me
      on why GC does keep the objects I can find no reference to marked.

   7) Up to now I have not been able to reproduce any of these effects
      with simple test programs. In my test cases all the objects always
      get collected as expected :-(

So after long nights of debugging I am pretty sure that my app is not
leaking - at least not significantly. Anyway, my app's memory is
growing, anyway, over time. I would have preferred to find some silly
bug in my program rather than not actually understanding what the hell
is going on.

Best regards,
Tom
----------------------------------------------------------------------------
Thomas Aeby, Kirchweg 40, 1735 Giffers, Switzerland, Voice : (+41)26
4180040
Internet: aeby@graeff.com                           PGP public key
available
----------------------------------------------------------------------------
One way to stop a runaway horse is to bet on him.