Re: Circumventing __USE_MALLOC and templatized containers

[Wolfgang Bangerth <wolfgang dot bangerth at iwr dot uni-heidelberg dot de>]

> Just so you know, at the moment, the __USE_MALLOC macro may still be
> defined as long as you do it right.  The correct way to use it is to
> edit c++config, follow all related commentary in that file and
> completely rebuild libstdc++

:-) That's no simpler than subclassing from std::vector, etc. If you ask
me, axe this option, probably very few people will be willing to use it.


> In an ideal world: threading policy would be run-time changeable;
> memory allocation policy would be run-time changeable.

Maybe that is not what is really necessary. It may be sufficient to choose
it once per program execution, or at link time (i.e. no recompilation
necessary, neither of libstdc++ not of user program).

> The current
> code base just doesn't allow that.  The designers of the current STL
> implementation we use chose top speed over ease of run-time
> configuration.  I don't think we can blame them for being unflexible
> given the root of C++, etc.

On the contrary, their aim for speed very much laid the basis for the
success of the STL.


> If someone came up with a method with near-zero overhead to allow
> global policy changes in these areas at run-time (instead of needing
> to rebuild libstdc++ or using a level of indirection in type selection
> in your application code), then I think it may be acceptable by the
> libstdc++-v3 gurus

OK, so let me make a proposal right out of my head, i.e. with all flaws
that first ideas have. Note that I'm no libstdc++ guru myself, and will
most probably not be able to implement this (though I'd be happy to help
with documenting this).

General idea (names only for illustrative purposes, no need to discuss
them now):

  namespace __internal {
    enum MemoryAllocationScheme {
      default_allocator, pool_based, malloc_based
    };

    // have a variable that designates the allocation scheme to be
    // used. note that it is const.
    extern const MemoryAllocationScheme memory_allocation_scheme;
  }


  // have standard allocator redirect to one of the existing allocators
  // based on the value of the variable above. note that all methods in
  // existing allocators seem to be static, so indirection is simple
  class new_default_allocator {
    static void method_1 () {
      switch (__internal::memory_allocation_scheme) {
        case default_allocator:  return old_default_allocator.method_1 ();
        case pool_based:         return poll_based_allocator.method_1 ();
        case malloc_based:       return malloc_based_allocator.method_1 ();
        default:                 abort();
    };

    // same with other methods
  };

The overhead for this one indirection is relatively minor, compared with
the total cost of memory allocation. Note that everything can be optimized
due to the tail-calls, and we can still keep the existing implementation
in the headers files, allowing for inlining of the called methods if that
is desired.

The question is now how to set the value of __internal::memory_allocation_scheme.
One idea would be to have a flag to the compiler, say -allocator=xxx,
where xxx from {default_allocator,pool_based,malloc_based}, and depending
on the flag, we link with either of a number of files, that each simply
contain

  namespace __internal {
    const MemoryAllocationScheme memory_allocation_scheme = malloc_based;
  };

or one of the other possible values. Alternatively, this action could be
triggered by the presence of -pthreads on the linker line, etc.


> To wrap up, the hard-coding of various policies is not there to make
> user's lives harder.  It is just the way the code base is until
> someone makes it more flexible.

That's very much understood, no blame intended, just the strive to improve
things.

Regards
  Wolfgang

-------------------------------------------------------------------------
Wolfgang Bangerth          email: wolfgang.bangerth@iwr.uni-heidelberg.de
                             www: http://gaia.iwr.uni-heidelberg.de/~wolf