[Bug libstdc++/16614] New: Excessive resource usage in __mt_alloc

["peturr02 at ru dot is" <gcc-bugzilla at gcc dot gnu dot org>]

In include/ext/mt_allocator.h:

  template<typename _Tp>
    class __mt_alloc
    {
[...]
      struct _Thread_record
      {
        // Points to next free thread id record. NULL if last record in list.
        _Thread_record* volatile        _M_next;

        // Thread id ranging from 1 to _S_max_threads.
        size_t                          _M_id;
      };
[...]
      if (__gthread_active_p())
        {
          const size_t __k = sizeof(_Thread_record) * _S_options._M_max_threads;
          __v = ::operator new(__k);
[...]
          // Initialize per thread key to hold pointer to
          // _S_thread_freelist.
          __gthread_key_create(&_S_thread_key, _S_destroy_thread_key);

The problem: This happens in the template __mt_alloc<_Tp>, so this code gets
run for each type for which __mt_alloc is instantiated. The default value of
_M_max_threads is 4096 and sizeof(_Thread_record) is 8 (on 32-bit systems)
so that means 32k per type. For a large program that can easily mean several
hundred megabytes.

The use of __gthread_key_create is even worse; there is only a limited number
of available keys for each program, so this places a hard upper limit on the
number of types that can be used with __mt_alloc. It also limits the number of
keys available for other purposes. The return value of __gthread_key_create
should be checked, since it may fail.

There doesn't seem to be any benefit to having _Thread_record depend on the
template parameter, it didn't do so until this change:

2003-12-23  Benjamin Kosnik  <bkoz@redhat.com>

	* include/ext/new_allocator.h (new_allocator): Proper allocator class.
	(__new_alloc): Delete.
	* include/ext/malloc_allocator.h (malloc_allocator): Same.
	(__malloc_alloc): Delete.
 	* include/ext/mt_allocator.h: Same, but weakly.
 	* include/ext/debug_allocator.h: Convert to the new style.
	* include/ext/pool_allocator.h: Use global new and delete directly.
	* include/backward/alloc.h: Don't inject malloc_allocator, or
	debug_allocator.
	* testsuite/ext/allocators.cc: Minimal fixups for usage of new
	classes.  Comment out tests with __pool_alloc for now.
	* testsuite/performance/allocator.cc: Same.

It seems that none of the other nested types or static members need the
template parameter either, and indeed most of the code seems to assume that
there will be only one copy of the static members (for example, a bucket of
chunks of size 8 will be created for all types, but this bucket will never be
used if sizeof(_Tp) > 8).

-- 
           Summary: Excessive resource usage in __mt_alloc
           Product: gcc
           Version: 3.4.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: libstdc++
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: peturr02 at ru dot is
                CC: gcc-bugs at gcc dot gnu dot org
 GCC build triplet: i686-pc-linux-gnu
  GCC host triplet: i686-pc-linux-gnu
GCC target triplet: i686-pc-linux-gnu


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=16614