RE: Re: [pylucene-dev] investigations

["Boehm, Hans" <hans dot boehm at hp dot com>]

Unfortunately, I'm neither a PyLucene nor a Python expert.

What does os.fork() do?  It calls Posix fork()?  Hopefully from
a single-threaded process?  (A process forked from a multithreaded
parent can do very little other than call exec.  Unfortunately,
libjava currently appears to violate that rule by calling
malloc.  I think that's a bug.)

It would be helpful to look at what happens if you define
the GC_PRINT_STATS environment variable.  That should give you
some more insight as to how the two cases differ.

There is leak debugging support, in libjava, though I'm
not sure whether it's currently fully functional.  To do so,
you need to rebuild the library with --enable-gc-debug,
and then

setenv GC_BACKTRACES 1 (or some small integer)

when you run.

Hans



> -----Original Message-----
> From: java-owner@gcc.gnu.org [mailto:java-owner@gcc.gnu.org] 
> On Behalf Of Yura Smolsky
> Sent: Friday, September 02, 2005 9:07 AM
> To: java@gcc.gnu.org
> Cc: Andi Vajda
> Subject: Fwd: Re: [pylucene-dev] investigations
> 
> 
> Hello, java@gcc.gnu.org.
> 
> I am user of PyLucene lib which was built with gcc/gcj.
> I have some interesting question.
> 
> Thanks in advance for the answer.
> 
> ---------- Forward Message ----------
> From:   Andi Vajda <vajda@osafoundation.org>
> To:     Yura Smolsky <info@altervision.biz>, 
> pylucene-dev@osafoundation.org
> CC:     
> Time:   Fri, 02 Sep 2005 10:58:11 -0500
> Subj:   [pylucene-dev] investigations
> Attach: <none>
> 
> 
> > I have found that PyLucene produces much more leaks under 
> linux when 
> > its run from forked process:
> >        newpid = os.fork()
> >        if newpid == 0:
> >            child_process()
> >        else:
> >            os.getpid()
> >
> > I run SimpleXMLRPCServer with PyLucene threads and perform some 
> > searches inside of them When I run script without forking leaks are 
> > like 20mb per 30 minutes and with forking they are about 
> 120mb per 30 
> > minutes.
> >
> > Any ideas why forking increases leaks?
> 
> No, but I bet Hans Boehm or someone at java@gnu.gcc.org could 
> shed some light 
> on this. Good find !
> 
> So far, these leaks have been only been reported on Linux, so 
> it's got to be 
> something between gcc/gcj and Linux...
> 
> Andi..
> 
> 
> 
> ---------- End of Forward ----------
> 
> Yura Smolsky,
> 
> 
>