RE: Re: [OT] Re: Controlling the garbage collector (GC) at RT?

["Boehm, Hans" <hans dot boehm at hp dot com>]

It's probably best to move this discussion to some linux kernel
group, since it no longer seems to be related to gcj.

I would try it on the newest available kernel, with as few other
applications
running as possible.  It is conceivable that the problem is

- Some other application which somehow convinces the kernel it needs
less
memory than it actually does.

- A temporary bug in 2.6.5.

- Some misbehaving kernel module specific to your configuration.

I think the basic mechanism should work if the /proc/sys file is there.
I assume overcommit_memory reads back as the value you echoed into it?

Hans


> -----Original Message-----
> From: java-owner@gcc.gnu.org [mailto:java-owner@gcc.gnu.org] 
> On Behalf Of Martin Egholm Nielsen
> Sent: Thursday, March 17, 2005 11:45 PM
> To: java@gcc.gnu.org
> Subject: Re: [OT] Re: Controlling the garbage collector (GC) at RT?
> 
> 
> Hi,
> 
> > I tried the attached program on Linux 2.6.7rc3 running on a 
> 4GB IA64 
> > machine with 1GB swap space.  It failed with a signal with 
> the default 
> > overcommit policy of 0.
> > 
> > After an
> > 
> > echo 2 > /proc/sys/vm/overcommit_memory
> > 
> > as root it failed with sbrk returning zero. My 
> interpretation is that 
> > the overcommit policy zero heuristic may need a bit of work, but 
> > things basically work as described.
> Not at my place :-)
> 
> > I repeated the experiment with an old X86 machine (256M mem + 512M 
> > swap) running 2.4.18, with basically identical results. 
> This seems to 
> > be a system configuration issue.
> So you think it requires some kernel configuration?
> 
> But then it's weird the kernel bothers to create the 
> /proc/sys/vm/overcommit_memory file, if I have it disabled somehow...
> 
> But I verified that it should work on an 2.6.5/i386...
> 
> // Martin
> 
> > 
> >>-----Original Message-----
> >>From: Martin Egholm Nielsen [mailto:martin@egholm-nielsen.dk]
> >>Sent: Thursday, March 17, 2005 1:15 AM
> >>To: Boehm, Hans
> >>Subject: [OT] Re: Controlling the garbage collector (GC) at RT?
> >>
> >>
> >>Hi Hans,
> >>
> >>
> >>>>>You also need to touch the resulting memory.  Try something like
> >>>>>char *v = sbrk( 1000000 ); for (char *p = v; p < p + 
> >>
> >>1000000; ++p)
> >>
> >>>>>*p = 42; in the loop.
> >>>>
> >>>>I'll try something similar tomorrow, thanks alot - 
> although I will 
> >>>>need modify the ever true statement "p < p + 1M" to "p < 
> v + 1M"...
> >>>
> >>>So, now I tried this:
> >>>
> >>>#include <unistd.h>
> >>>
> >>>int main( int i )
> >>>{
> >>>  while ( 1 ) {
> >>>    char *v = sbrk( 1000000 );
> >>>    char *p;
> >>>    for (*p = v; p < v + 1000000; ++p) {
> >>>      *p = 42;
> >>>    } // for
> >>>
> >>>    printf( "%x\n\n", v );
> >>>  } // while
> >>>} // main
> >>>
> >>>And still nothing happens with the memory consumption, 
> although the 
> >>>pointer ends up at 0xffffffff...
> >>
> >>Nooow, after modifying the program to
> >>
> >>for ( p = v; ...
> >>
> >>it "works" - the application is terminated by the kernel. This goes 
> >>regardless what [0,1,2] is echo'ed into 
> >>"/proc/sys/vm/overcommit_memory"...
> >>The kernel reports:
> >>
> >>Out of Memory: Killed process 36 (sbrktest)
> >>
> >>But what should the outcome really had been?
> >>
> >>I tried the same on an elder kernel on a different target -
> >>2.4.17 and 
> >>i386 - and the same outcome...
> >>
> >>BR,
> >>  Martin Egholm
> >>
> >>
> >>>>>>-----Original Message-----
> >>>>>>From: java-owner@gcc.gnu.org [mailto:java-owner@gcc.gnu.org] On
> >>>>>>Behalf Of Martin Egholm Nielsen
> >>>>>>Sent: Wednesday, February 23, 2005 12:04 AM
> >>>>>>To: java@gcc.gnu.org
> >>>>>>Subject: Re: Controlling the garbage collector (GC) at RT?
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>It seems to me that the only problem here is that this
> >>>>>>
> >>>>>>still seems to
> >>>>>>
> >>>>>>>happen with overcommit-accounting set to 2.
> >>>>>>>I would expect that you can reproduce this problem with a
> >>>>>>
> >>>>>>program that
> >>>>>>
> >>>>>>>alternately allocates a few MB with sbrk, and then touches the
> >>>>>>>allocated memory.  If you can't, there's something really 
> >>>>>>
> >>>>>>weird going
> >>>>>>
> >>>>>>>on here.  If you can, it'll give you a test case for the kernel
> >>>>>>>people.
> >>>>>>
> >>>>>>I tried the following:
> >>>>>>
> >>>>>>#include <unistd.h>
> >>>>>>
> >>>>>>int main( int i )
> >>>>>>{
> >>>>>>  while ( 1 ) {
> >>>>>>    void *v = sbrk( 100000 );
> >>>>>>    // sleep( 1 );
> >>>>>>  } // while
> >>>>>>} // main
> >>>>>>
> >>>>>>But that doesn't result in anything - the memory usage for the
> >>>>>>application does not grow...
> >>>>>>Are there anything else I should do to allocate the memory?
> >>>>>>
> >>>>>>BR,
> >>>>>> Martin
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>Hans
> >>>>>>>
> >>>>>>>On Wed, 16 Feb 2005, Martin Egholm Nielsen wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>>Hi Hans,
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>>>It would indeed be interesting to know why the Linux kernel
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>>>kills the application rather than returning failure.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>>Sure, but how to do that? Any guidelines?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>What do you see on the console?  Anything in the system log?
> >>>>>>>>>Does strace tell you anything?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>Below is the last part of "strace -f -F -i -v". It doesn't
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>really look
> >>>>>>
> >>>>>>
> >>>>>>>>like there's anything of value?
> >>>>>>>>
> >>>>>>>>// Martin
> >>>>>>>>
> >>>>>>>>[pid    75] [0f833558] write(1, "*** MEM CHUNK TAKEN: 
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>8388608\n", 29***
> >>>>>>
> >>>>>>
> >>>>>>>>MEM CHUNK TAKEN: 8388608
> >>>>>>>>) = 29
> >>>>>>>>[pid    75] [0f839834] brk(0x12d15000)  = 0x12d15000
> >>>>>>>>[pid    75] [0f839834] brk(0x12d25000)  = 0x12d25000
> >>>>>>>>[pid    75] [0f81126c] getpid()         = 75
> >>>>>>>>[pid    75] [0f799444] kill(77, SIGPWR <unfinished ...>
> >>>>>>>>[pid    76] [0f840e2c] <... poll resumed> [{fd=3, 
> >>
> >>events=POLLIN,
> >>
> >>>>>>>>revents=POLLIN}
> >>>>>>>>], 1, 2000) = 1
> >>>>>>>>[pid    75] [0f799444] <... kill resumed> ) = 0
> >>>>>>>>[pid    76] [0f81127c] getppid()        = 75
> >>>>>>>>[pid    76] [0f833548] read(3,
> >>>>>>>>"\20\7/\234\0\0\0\4\17\374$0\20\7/\240$\0\0B\17\3
> >>>>>>>>72j(\177"..., 148) = 148
> >>>>>>>>[pid    76] [0f840e2c] poll( <unfinished ...>
> >>>>>>>>[pid    75] [0f799444] kill(77, SIGXCPU) = 0
> >>>>>>>>[pid    75] [0f839834] brk(0x13525000)  = 0x13525000
> >>>>>>>>[pid    75] [0f833558] write(1, "*** MEM CHUNK TAKEN: 
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>8388608\n", 29***
> >>>>>>
> >>>>>>
> >>>>>>>>MEM CHUNK TAKEN: 8388608
> >>>>>>>>) = 29
> >>>>>>>>[pid    75] [0f839834] brk(0x13535000)  = 0x13535000
> >>>>>>>>[pid    75] [0f839834] brk(0x13545000)  = 0x13545000
> >>>>>>>>[pid    75] [0f839834] brk(0x13d45000)  = 0x13d45000
> >>>>>>>>[pid    75] [0f833558] write(1, "*** MEM CHUNK TAKEN: 
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>8388608\n", 29***
> >>>>>>
> >>>>>>
> >>>>>>>>MEM CHUNK TAKEN: 8388608
> >>>>>>>>) = 29
> >>>>>>>>[pid    76] [0f840e2c] <... poll resumed> [{fd=3, 
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>events=POLLIN}], 1,
> >>>>>>
> >>>>>>
> >>>>>>>>2000) = 0
> >>>>>>>>[pid    76] [0f840e2c] --- SIGTERM (Terminated) ---
> >>>>>>>>#
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>
> >>>
> > 
> 
>