Re: [RFC] allocator reorg

[Joachim Schurig <js at logoport dot net>]

Hi,

I am using the __mt_alloc allocator since 2 months as the default 
allocator with libstdc++ and did some debugging on it.

I am happy about your implementation, it looks clean to me.

However, I would like to emphasize the point that there should be an 
easy way to make the __mt_alloc allocator the default allocator when 
running on SMP machines. I currently patch stl_alloc.h to typedef 
__mt_alloc as __alloc.

As you probably know, due to it's global lock the runtime behaviour of 
the (standard) __pool_alloc on Intel SMP machines is next to unusable. I 
tried to use it with a multithreaded server application on a Quad PIII 
SMP machine, but it was more efficient to simply switch off 3 of the 4 
CPUs. If you'd like me to, I can run some tests again and send you the 
results. It should be noted that parts of the bad performance were due 
to an inefficient capacity strategy in the basic_string implementation, 
see my other today mail on a solution for that.

With __mt_alloc, performance was linear and stable.

In my perception, the standard library itself should decide which 
allocator suits best to its runtime environment, if there is no explicit 
declaration by the programmer. Therefore, I do not really like the 
GLIBCPP_FORCE_NEW switch (or an extension to MT_ALLOC), as it leaves the 
allocator decision to the user of an application, not even to the 
programmer. This may be the worst scenario.

The linux pthread lib uses something like the following code to 
determine if it runs on an SMP kernel. I have no idea how portable the 
sysctl() call is, the parsing of the version string as last resort 
definitely isn't portable. However, with something like this the 
libstdc++ could find out if it runs on SMP, and in that case delegate 
calls to the standard allocator to __mt_alloc, which would always 
guarantee a better runtime behaviour than using __pool_alloc.

Best regards,

Joachim Schurig

/* Test whether the machine has more than one processor.  This is not the
  best test but good enough.  More complicated tests would require `malloc'
  which is not available at that time.  */

bool is_smp_system(void)
{
 static const int sysctl_args[] = { CTL_KERN, KERN_VERSION };
 char buf[512];
 size_t reslen = sizeof (buf);

 /* Try reading the number using `sysctl' first.  */
 if (sysctl ((int *) sysctl_args,
       sizeof (sysctl_args) / sizeof (sysctl_args[0]),
       buf, &reslen, NULL, 0) < 0)
   {
     /* This was not successful.  Now try reading the /proc filesystem.  */
     int fd = open ("/proc/sys/kernel/version", O_RDONLY);
     if (fd < 0
     || (reslen = read (fd, buf, sizeof (buf) - 1)) <= 0) {
   /* This also didn't work.  We give up and say it's a UP machine.  */
   buf[0] = '\0';
     } else {
         buf[reslen] = '\0';
     }

     if (fd >= 0) close (fd);
   }

 // expect an arbitrary length string returned above that does not 
include a 0 trailer
 buf[511] = 0;

 return strstr (buf, "SMP") != NULL;
}



Benjamin Kosnik wrote:

> This patch adds the MT allocator graciously donated by Stefan Olsson and
> Ola Rönnerup. It also re-organizes the allocator code to improve
> legibility and to make it more modular.
>
> I think this is the right way to go, and would like to check this in.
> However, I'll wait for 24 hrs for comments. As people may be aware, the
> 3.4 branch creation deadline is looming, so CVS intensive work should be
> done by the end of the week.
>
> Things that may be done after this patch is in:
>
> 1) splitting __gnu_cxx into public/private or adding nested namespaces
> for this. Please discuss this on a separate thread. 
>
> 2) removing some of the allocator_traits.h bits. 
>
> 3) trying to figure out a better way to switch default allocators, ie
> extensions to _GLIBCPP_FORCE_NEW.
>
> tested x86/linux
>
> -benjamin
>
>