Re: _set_new_handler vs. set_new_handler

[Marcel Lanz <marcel dot lanz at ds9 dot ch>]

> Additionally I just noticed that I probably don't understand the overall
> mechanism. If you call "new" and this operation fails due the lack of
> memory, then the given new-handler is called. In this new-handler you
> are freeing a bunch of panic memory. The successional calls to new then
> will probably work fine, since free memory is available. But what's
> about the original new-call, which caused the handler to be executed? It
> will still fail and return NULL, won't it? Or is there any kind of
> retry-machanism I don't know about? The C runtime will not retry to
> allocate the memory after a call to the new-handler, will it?

in GCC ::operator new is implemented like this:

void *
operator new (std::size_t sz) throw (std::bad_alloc)
{
  void *p = (void *) malloc (sz);
  while (p == 0)
    {
      new_handler handler = __new_handler;
      if (! handler)
        std::abort();
      handler ();
      p = (void *) malloc (sz);
    }
  return p;
}

if I read this ::operator new will call "handler()" as long as there is one and then again call
malloc. if malloc return nothing, a new handler is fetched and called if it exists.

each handler has to set a new handler, so that the next time, we run out of memory, 
another handler can be called.

I think memory is not wastet. its released on request. sure If I held back a big chunk of it and
malloc tries to get a small amount of memory, the handler will release more than needed, but the application
will know, that someone called for more memory than (MAX_MEM - preallocated_mem). apply this to multiple steps
and you can track the way to disaster (in terms of memory exhaustion)

for sure, If my app allocates that much memory I can never handle, I'm lost. But small growing memory
leaks can be monitored, this way if I know how much memory my system should use during runtime.

see ECPP Item 7: Be prepared for out-of-memory conditions.

> There are two issues I don't understand: 
> You say this solution doesn't waste memory. But in fact the preallocated
> memory is lost for the rest of the system and under normal conditions
> the application will not use that panic memory at all. I think this
> could be called "wasting" the memory, isn't it?
in an embedded system with one big heap, no
otherwise, perhaps yes. if the OS has virtual memory system no

The Linux Kernel has memory flagged as "commited", which is the amount of memory that will be used if the allocated memory
would be used. so you can allocate memory, but if its not used (writing to it), you can allocate more memory
than you have on your box.

try this:

#include <iostream>
#include <unistd.h

int main(void)
{
    char* m1 = new char[1024*1024*1024];
    char* m2 = new char[1024*1024*1024];

    while(1) {pause();}
        
    return 0;
}

the application will have 2GB of memory allocated, but the system (GNU/Linux in my case) will mark this memory a virtual, and it won't
allocate it on RAM or swap until you use it.

this brings me to the conclusion, that you don't need a new_handler fallback if the operating system can handle
out of memory exceptions. another thing then I'd look at, is a mechanism to limit an applicaiotns memory (virtually and physically) by
the operating system.

eigther, I think memory exhaustion should not occur on a well designed application and environment :-)

and I think we are mislocated here in this mailinglist ? arent' we ?

marcel