RE: [OT] Re: Controlling the garbage collector (GC) at RT?

["Boehm, Hans" <hans dot boehm at hp dot com>]

I tried the attached program on Linux 2.6.7rc3 running on a 4GB IA64
machine with 1GB swap space.  It failed with a signal with the default
overcommit policy of 0.

After an

echo 2 > /proc/sys/vm/overcommit_memory

as root it failed with sbrk returning zero.  My interpretation is that
the overcommit policy zero heuristic may need a bit of work, but things
basically work as described.

I repeated the experiment with an old X86 machine (256M mem + 512M swap)
running 2.4.18, with basically identical results.

This seems to be a system configuration issue.

Hans

> -----Original Message-----
> From: Martin Egholm Nielsen [mailto:martin@egholm-nielsen.dk] 
> Sent: Thursday, March 17, 2005 1:15 AM
> To: Boehm, Hans
> Subject: [OT] Re: Controlling the garbage collector (GC) at RT?
> 
> 
> Hi Hans,
> 
> >>> You also need to touch the resulting memory.  Try something like 
> >>> char *v = sbrk( 1000000 ); for (char *p = v; p < p + 
> 1000000; ++p) 
> >>> *p = 42; in the loop.
> >> I'll try something similar tomorrow, thanks alot - although I will
> >> need modify the ever true statement "p < p + 1M" to "p < v + 1M"...
> > So, now I tried this:
> > 
> > #include <unistd.h>
> > 
> > int main( int i )
> > {
> >   while ( 1 ) {
> >     char *v = sbrk( 1000000 );
> >     char *p;
> >     for (*p = v; p < v + 1000000; ++p) {
> >       *p = 42;
> >     } // for
> > 
> >     printf( "%x\n\n", v );
> >   } // while
> > } // main
> > 
> > And still nothing happens with the memory consumption, although the
> > pointer ends up at 0xffffffff...
> Nooow, after modifying the program to
> 
> for ( p = v; ...
> 
> it "works" - the application is terminated by the kernel.
> This goes regardless what [0,1,2] is echo'ed into 
> "/proc/sys/vm/overcommit_memory"...
> The kernel reports:
> 
> Out of Memory: Killed process 36 (sbrktest)
> 
> But what should the outcome really had been?
> 
> I tried the same on an elder kernel on a different target - 
> 2.4.17 and 
> i386 - and the same outcome...
> 
> BR,
>   Martin Egholm
> 
> >>>> -----Original Message-----
> >>>> From: java-owner@gcc.gnu.org [mailto:java-owner@gcc.gnu.org] On 
> >>>> Behalf Of Martin Egholm Nielsen
> >>>> Sent: Wednesday, February 23, 2005 12:04 AM
> >>>> To: java@gcc.gnu.org
> >>>> Subject: Re: Controlling the garbage collector (GC) at RT?
> >>>>
> >>>>
> >>>>
> >>>>> It seems to me that the only problem here is that this 
> >>>> still seems to
> >>>>> happen with overcommit-accounting set to 2.
> >>>>> I would expect that you can reproduce this problem with a 
> >>>> program that
> >>>>> alternately allocates a few MB with sbrk, and then touches the 
> >>>>> allocated memory.  If you can't, there's something really 
> >>>> weird going
> >>>>> on here.  If you can, it'll give you a test case for the kernel 
> >>>>> people.
> >>>> I tried the following:
> >>>>
> >>>> #include <unistd.h>
> >>>>
> >>>> int main( int i )
> >>>> {
> >>>>   while ( 1 ) {
> >>>>     void *v = sbrk( 100000 );
> >>>>     // sleep( 1 );
> >>>>   } // while
> >>>> } // main
> >>>>
> >>>> But that doesn't result in anything - the memory usage for the 
> >>>> application does not grow...
> >>>> Are there anything else I should do to allocate the memory?
> >>>>
> >>>> BR,
> >>>>  Martin
> >>>>
> >>>>
> >>>>> Hans
> >>>>>
> >>>>> On Wed, 16 Feb 2005, Martin Egholm Nielsen wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>>> Hi Hans,
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>>> It would indeed be interesting to know why the Linux kernel
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>>> kills the application rather than returning failure.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>> Sure, but how to do that? Any guidelines?
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> What do you see on the console?  Anything in the system log?  
> >>>>>>> Does strace tell you anything?
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Below is the last part of "strace -f -F -i -v". It doesn't 
> >>>>
> >>>>
> >>>>
> >>>> really look
> >>>>
> >>>>>> like there's anything of value?
> >>>>>>
> >>>>>> // Martin
> >>>>>>
> >>>>>> [pid    75] [0f833558] write(1, "*** MEM CHUNK TAKEN: 
> >>>>
> >>>>
> >>>>
> >>>> 8388608\n", 29***
> >>>>
> >>>>>> MEM CHUNK TAKEN: 8388608
> >>>>>> ) = 29
> >>>>>> [pid    75] [0f839834] brk(0x12d15000)  = 0x12d15000
> >>>>>> [pid    75] [0f839834] brk(0x12d25000)  = 0x12d25000
> >>>>>> [pid    75] [0f81126c] getpid()         = 75
> >>>>>> [pid    75] [0f799444] kill(77, SIGPWR <unfinished ...>
> >>>>>> [pid    76] [0f840e2c] <... poll resumed> [{fd=3, 
> events=POLLIN,
> >>>>>> revents=POLLIN}
> >>>>>> ], 1, 2000) = 1
> >>>>>> [pid    75] [0f799444] <... kill resumed> ) = 0
> >>>>>> [pid    76] [0f81127c] getppid()        = 75
> >>>>>> [pid    76] [0f833548] read(3,
> >>>>>> "\20\7/\234\0\0\0\4\17\374$0\20\7/\240$\0\0B\17\3
> >>>>>> 72j(\177"..., 148) = 148
> >>>>>> [pid    76] [0f840e2c] poll( <unfinished ...>
> >>>>>> [pid    75] [0f799444] kill(77, SIGXCPU) = 0
> >>>>>> [pid    75] [0f839834] brk(0x13525000)  = 0x13525000
> >>>>>> [pid    75] [0f833558] write(1, "*** MEM CHUNK TAKEN: 
> >>>>
> >>>>
> >>>>
> >>>> 8388608\n", 29***
> >>>>
> >>>>>> MEM CHUNK TAKEN: 8388608
> >>>>>> ) = 29
> >>>>>> [pid    75] [0f839834] brk(0x13535000)  = 0x13535000
> >>>>>> [pid    75] [0f839834] brk(0x13545000)  = 0x13545000
> >>>>>> [pid    75] [0f839834] brk(0x13d45000)  = 0x13d45000
> >>>>>> [pid    75] [0f833558] write(1, "*** MEM CHUNK TAKEN: 
> >>>>
> >>>>
> >>>>
> >>>> 8388608\n", 29***
> >>>>
> >>>>>> MEM CHUNK TAKEN: 8388608
> >>>>>> ) = 29
> >>>>>> [pid    76] [0f840e2c] <... poll resumed> [{fd=3, 
> >>>>
> >>>>
> >>>>
> >>>> events=POLLIN}], 1,
> >>>>
> >>>>>> 2000) = 0
> >>>>>> [pid    76] [0f840e2c] --- SIGTERM (Terminated) ---
> >>>>>> #
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> >>
> > 
> > 
> 
>

Attachment: exhaust_mem.c
Description: exhaust_mem.c