Re: [OT] Re: Controlling the garbage collector (GC) at RT?

[Martin Egholm Nielsen <martin at egholm-nielsen dot dk>]

Hi again,

> I tried the attached program on Linux 2.6.7rc3 running on a 4GB IA64
> machine with 1GB swap space.  It failed with a signal with the default
> overcommit policy of 0.
>
> After an
>
> echo 2 > /proc/sys/vm/overcommit_memory
>
> as root it failed with sbrk returning zero.  My interpretation is that
> the overcommit policy zero heuristic may need a bit of work, but things
> basically work as described.
>
> I repeated the experiment with an old X86 machine (256M mem + 512M swap)
> running 2.4.18, with basically identical results.
>
> This seems to be a system configuration issue.
As far I can see, this may be a known problem with systems where swap is 
disabled:

http://www.ussg.iu.edu/hypermail/linux/kernel/0208.2/1030.html

I'm taking the thread somewhere else now - thanks for the help Hans!

BR,
 Martin Egholm

> Hans
>
>
> > -----Original Message-----
> > From: Martin Egholm Nielsen [mailto:martin@egholm-nielsen.dk] 
> > Sent: Thursday, March 17, 2005 1:15 AM
> > To: Boehm, Hans
> > Subject: [OT] Re: Controlling the garbage collector (GC) at RT?
> >
> >
> > Hi Hans,
> >
> >
> > > > > You also need to touch the resulting memory.  Try something like 
> > > > > char *v = sbrk( 1000000 ); for (char *p = v; p < p + 
> >
> > 1000000; ++p) 
> >
> > > > > *p = 42; in the loop.
> > > >
> > > > I'll try something similar tomorrow, thanks alot - although I will
> > > > need modify the ever true statement "p < p + 1M" to "p < v + 1M"...
> > >
> > > So, now I tried this:
> > >
> > > #include <unistd.h>
> > >
> > > int main( int i )
> > > {
> > >  while ( 1 ) {
> > >    char *v = sbrk( 1000000 );
> > >    char *p;
> > >    for (*p = v; p < v + 1000000; ++p) {
> > >      *p = 42;
> > >    } // for
> > >
> > >    printf( "%x\n\n", v );
> > >  } // while
> > > } // main
> > >
> > > And still nothing happens with the memory consumption, although the
> > > pointer ends up at 0xffffffff...
> >
> > Nooow, after modifying the program to
> >
> > for ( p = v; ...
> >
> > it "works" - the application is terminated by the kernel.
> > This goes regardless what [0,1,2] is echo'ed into 
> > "/proc/sys/vm/overcommit_memory"...
> > The kernel reports:
> >
> > Out of Memory: Killed process 36 (sbrktest)
> >
> > But what should the outcome really had been?
> >
> > I tried the same on an elder kernel on a different target - 
> > 2.4.17 and 
> > i386 - and the same outcome...
> >
> > BR,
> >  Martin Egholm
> >
> >
> > > > > > -----Original Message-----
> > > > > > From: java-owner@gcc.gnu.org [mailto:java-owner@gcc.gnu.org] On 
> > > > > > Behalf Of Martin Egholm Nielsen
> > > > > > Sent: Wednesday, February 23, 2005 12:04 AM
> > > > > > To: java@gcc.gnu.org
> > > > > > Subject: Re: Controlling the garbage collector (GC) at RT?
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > > It seems to me that the only problem here is that this 
> > > > > >
> > > > > > still seems to
> > > > > >
> > > > > > > happen with overcommit-accounting set to 2.
> > > > > > > I would expect that you can reproduce this problem with a 
> > > > > >
> > > > > > program that
> > > > > >
> > > > > > > alternately allocates a few MB with sbrk, and then touches the 
> > > > > > > allocated memory.  If you can't, there's something really 
> > > > > >
> > > > > > weird going
> > > > > >
> > > > > > > on here.  If you can, it'll give you a test case for the kernel 
> > > > > > > people.
> > > > > >
> > > > > > I tried the following:
> > > > > >
> > > > > > #include <unistd.h>
> > > > > >
> > > > > > int main( int i )
> > > > > > {
> > > > > >  while ( 1 ) {
> > > > > >    void *v = sbrk( 100000 );
> > > > > >    // sleep( 1 );
> > > > > >  } // while
> > > > > > } // main
> > > > > >
> > > > > > But that doesn't result in anything - the memory usage for the 
> > > > > > application does not grow...
> > > > > > Are there anything else I should do to allocate the memory?
> > > > > >
> > > > > > BR,
> > > > > > Martin
> > > > > >
> > > > > >
> > > > > >
> > > > > > > Hans
> > > > > > >
> > > > > > > On Wed, 16 Feb 2005, Martin Egholm Nielsen wrote:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > Hi Hans,
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > > > > It would indeed be interesting to know why the Linux kernel
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > > > > kills the application rather than returning failure.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > > > Sure, but how to do that? Any guidelines?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > What do you see on the console?  Anything in the system log?  
> > > > > > > > > Does strace tell you anything?
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Below is the last part of "strace -f -F -i -v". It doesn't 
> > > > > >
> > > > > >
> > > > > >
> > > > > > really look
> > > > > >
> > > > > >
> > > > > > > > like there's anything of value?
> > > > > > > >
> > > > > > > > // Martin
> > > > > > > >
> > > > > > > > [pid    75] [0f833558] write(1, "*** MEM CHUNK TAKEN: 
> > > > > >
> > > > > >
> > > > > >
> > > > > > 8388608\n", 29***
> > > > > >
> > > > > >
> > > > > > > > MEM CHUNK TAKEN: 8388608
> > > > > > > > ) = 29
> > > > > > > > [pid    75] [0f839834] brk(0x12d15000)  = 0x12d15000
> > > > > > > > [pid    75] [0f839834] brk(0x12d25000)  = 0x12d25000
> > > > > > > > [pid    75] [0f81126c] getpid()         = 75
> > > > > > > > [pid    75] [0f799444] kill(77, SIGPWR <unfinished ...>
> > > > > > > > [pid    76] [0f840e2c] <... poll resumed> [{fd=3, 
> >
> > events=POLLIN,
> >
> > > > > > > > revents=POLLIN}
> > > > > > > > ], 1, 2000) = 1
> > > > > > > > [pid    75] [0f799444] <... kill resumed> ) = 0
> > > > > > > > [pid    76] [0f81127c] getppid()        = 75
> > > > > > > > [pid    76] [0f833548] read(3,
> > > > > > > > "\20\7/\234\0\0\0\4\17\374$0\20\7/\240$\0\0B\17\3
> > > > > > > > 72j(\177"..., 148) = 148
> > > > > > > > [pid    76] [0f840e2c] poll( <unfinished ...>
> > > > > > > > [pid    75] [0f799444] kill(77, SIGXCPU) = 0
> > > > > > > > [pid    75] [0f839834] brk(0x13525000)  = 0x13525000
> > > > > > > > [pid    75] [0f833558] write(1, "*** MEM CHUNK TAKEN: 
> > > > > >
> > > > > >
> > > > > >
> > > > > > 8388608\n", 29***
> > > > > >
> > > > > >
> > > > > > > > MEM CHUNK TAKEN: 8388608
> > > > > > > > ) = 29
> > > > > > > > [pid    75] [0f839834] brk(0x13535000)  = 0x13535000
> > > > > > > > [pid    75] [0f839834] brk(0x13545000)  = 0x13545000
> > > > > > > > [pid    75] [0f839834] brk(0x13d45000)  = 0x13d45000
> > > > > > > > [pid    75] [0f833558] write(1, "*** MEM CHUNK TAKEN: 
> > > > > >
> > > > > >
> > > > > >
> > > > > > 8388608\n", 29***
> > > > > >
> > > > > >
> > > > > > > > MEM CHUNK TAKEN: 8388608
> > > > > > > > ) = 29
> > > > > > > > [pid    76] [0f840e2c] <... poll resumed> [{fd=3, 
> > > > > >
> > > > > >
> > > > > >
> > > > > > events=POLLIN}], 1,
> > > > > >
> > > > > >
> > > > > > > > 2000) = 0
> > > > > > > > [pid    76] [0f840e2c] --- SIGTERM (Terminated) ---
> > > > > > > > #