Re: Circumventing __USE_MALLOC and templatized containers

[Loren James Rittle <rittle at latour dot rsch dot comm dot mot dot com>]

>> Thanks.  Imho, the need to replace allocators has been
>> underestimated or even neglected.  I've had this problem
>> too and found no satisfactory solution: it is not possible
>> to replace the (default) allocator without practically
>> having to rewrite the STL and ending up with a totally
>> incompatible set of STL containers etc.

> That's my opinion as well. I am led to think that this wholesale
> replacement of one allocator by another is not what was intended.
> In my view, selecting between malloc-based or other allocators should be
> possible either by a mechanism like __USE_MALLOC, or, as I understand the
> ABI concerns that were raised, by a special flag that is set once per
> program execution.

For the record, at least in terms of run-time configuration
flexibility, I actually agree with you guys more or less (I did read
what Carlo had to do in his case based on the method Phil and I
suggest but without looking more closely than I did, I have no idea if
he had to do all that work or not --- I admit that I didn't expect to
have to override all the operators, etc).

Just so you know, at the moment, the __USE_MALLOC macro may still be
defined as long as you do it right.  The correct way to use it is to
edit c++config, follow all related commentary in that file and
completely rebuild libstdc++ (since you changed the ABI your local
libstdc++.so and .a may not link or, more likely, always work properly
against code compiled the other way).  As much as I want to completely
axe it, I would not even propose axing it (at least until one more
improvement happens ;-).  The real point of all my past ranting and
raving on this issue is that incorrect use of __USE_MALLOC macro is a
real source of bug reports that I studied over the years and I would
like to stop users from hanging themselves if possible (if for no
other reason than to reduce gcc bug reports ;-).  I am not the one
that made the library work this way. I am just the current message
bearer.

In an ideal world: threading policy would be run-time changeable;
memory allocation policy would be run-time changeable.  The current
code base just doesn't allow that.  The designers of the current STL
implementation we use chose top speed over ease of run-time
configuration.  I don't think we can blame them for being unflexible
given the root of C++, etc.

If someone came up with a method with near-zero overhead to allow
global policy changes in these areas at run-time (instead of needing
to rebuild libstdc++ or using a level of indirection in type selection
in your application code), then I think it may be acceptable by the
libstdc++-v3 gurus (IMHO but I only have one limited vote and I am
merely a tweaker not a library guru).  The trick will be getting
near-zero overhead.  One reason you will have a problem making this
change without introducing extra overhead is that you will have to
convert many things now defined in "open" headers behind "closed"
firewall headers.  A lot of stuff that now gets expanded inline (one
of the keys of C++ code run-time speed verses other languages) will be
moved into another compilation unit.  Perhaps one could even make an
argument for a change even if the overhead was not near-zero.  For
example, if tuning an application became easier and it became possible
to actually improve overall application performance without users have
to recompile their entire application, then you might be able to sell
it on that basis.  Designing the change; actually prototyping/doing
it; and profiling it against today's default model would be the only
way to get there.  I think it is a non-trivial amount of work to do
right.  It will involve undoing some of the optimizations in the
fully-baked STL implementation.

To wrap up, the hard-coding of various policies is not there to make
user's lives harder.  It is just the way the code base is until
someone makes it more flexible.

Regards,
Loren