Re: Pthread options (was: Re: Memory allocators / STL)

[Stefan Olsson <stefan at noname4us dot com>]

Here are my latest findings:

I have just recompiled the application with GCC 3.0.2 and the result without using
__USE_MALLOC is the same as before (some strange kind of internal race condition or
something to do with the allocator).
After recompiling GCC/libstdc++ with the __USE_MALLOC flag set in
include/bits/c++config the application works, but slower than it should/could.

The platform is Slackware 8.0 running on a dual PIII (HP lpr2000) machine, kernel
2.4.12, thread model autodetected to posix (it's not possible to set it to pthreads
- I am not sure if this is correct or not, but the compilation aborts with messages
about a missing thr-... file)

The application relies heavily on STL and uses the C mysql API (with home brewn C++
wrappers).

The sympton (when not compiled using __USE_MALLOC) is that the application works
fine when only one thread is running (or atleast so it seems from looking at the
response times) but when the internal STL caching memoryallocator is used the
performance drops rapidly and the system CPU usage goes up to well over 50%

Please give me some input on what more information is needed in order to find this
bug...

Brgds

/Stefan Olsson, CTO @ Noname4us

Loren James Rittle wrote:

> >>> - Quite a few discussions on lists have been about __USE_MALLOC and other
>
> >> There should not be anything required except using -pthread. You
> >> shouldn't have to use __USE_MALLOC at all.
>
> Your questions are covered in the libstdc++-v3 FAQ.  Although you
> might need to read between the lines a bit to answer your exact
> questions thus I will take a crack at answering them directly.
>
> [What about -D__USE_MALLOC?]
>
> Related to threading or otherwise, the current recommendation is that
> users not add any macro defines on the command line to enable features
> out of libstdc++-v3.  There is no condition under which it will help
> you without causing other issues to perhaps raise up (possible
> linkage/ABI problems).
>
> In particular, __USE_MALLOC should only be added to a libstdc++-v3
> configuration file, include/bits/c++config (where such user action is
> cautioned against), and the entire library should be rebuilt.  If you
> do not, then you might be violating the one-definition rule of C/C++
> and you might cause yourself untold problems.
>
> Also, I will tell you that I personally performance tested the
> implication of enabling __USE_MALLOC against (threaded and
> non-threaded) application code I have here.  The slow down for heavy
> STL container code was incredible.  It is possible that my code is not
> representative, but others were complaining about a "massive" slowdown
> in container speed between 2.95 and pre-3.0 libstdc++-v3 releases so I
> think not.  The results were posted back when we flipped the default
> configuration of libstdc++-v3 just before gcc 3.0 was released to
> match that of libstdc++-v2 as was shipped with gcc 2.95.X.
>
> If you find any platform where gcc reports a threading model other
> than none and where libstdc++-v3 builds a buggy container allocator
> when used with threads unless you define __USE_MALLOC, I want to hear
> about it ASAP.  In the past, correctness was the main reason people
> were led to believe that they should define __USE_MALLOC when using
> threads.
>
> > What about -D_REENTRANT?
>
> libstdc++-v3 headers themselves do not directly require _REENTRANT on
> any platform.  If your system's libc headers require a special macro
> to be defined to enable proper compilation of threaded-code, then you
> will need to consult your local documentation.  I can tell you that my
> platform used to require that special macro to be defined but today,
> no macros need to be defined to get access to thread-safe libc
> headers.
>
> > I was under the impression that the requirements for pthreads were
> > -D_REENTRANT (or the equivalent #define in the code) at compile time and
> > -lpthread at link time. I've never heard of this -pthread compile time
> > option, and neither has the GCC manual. Could you explain please?
>
> This is a very non-standardized area of gcc.  Some ports support a
> special flag (the spelling isn't even standardized yet) to add all
> required macros to a compilation and link-library additions and/or
> replacements at link time.  The documentation is weak.  Here is a
> quick summary from memory to display how ad hoc this is (some efforts
> have occurred to rationalize this stuff: I and others have come close
> to understanding enough to attempt to create a new order of thing, but
> not quite yet):
>
> On solaris, both -pthreads and -threads (with subtly different
> meanings) are honored.  On OSF, -pthread and -threads (with subtly
> different meanings) are honored.  On Linux/i386, -pthread is honored
> with a meaning that matches your impression on the correct way to
> ensure proper thread support.  FreeBSD supports -pthread.  Some other
> ports use other switches.  AFAIK, none of this is properly documented
> anywhere other than in ``gcc -dumpspecs''.  This situation existed
> before I became a gcc developer so I don't know any more of the history.
>
> I am remiss in that I didn't finish improving documentation for
> libstdc++-v3 before 3.0.2 release.  Since I basically just wrote the
> remaining piece above, expect a posting to libstdc++ with an update of
> the FAQ I promised some time ago.
>
> Regards,
> Loren
> --
> Loren J. Rittle
> Senior Staff Software Engineer, Distributed Object Technology Lab
> Networks and Infrastructure Research Lab (IL02/2240), Motorola Labs
> rittle@rsch.comm.mot.com, KeyID: 2048/ADCE34A5, FDC0292446937F2A240BC07D42763672

--
Military intelligence is a contradiction in terms.