Re: Application hangs with mutliple threads in atomicity functions

[Andrew Haley <aph-gcc at littlepinkcloud dot COM>]

Chad Attermann writes:
 > 
 > I am a first-time poster to the gcc lists and not intimately
 > familiar with compilers so please understand if I don't offer all
 > of the relevant information up-front. I am in the process of
 > porting a large application from gcc 2.95.3 with libstdc++ 2.10.0
 > to gcc 3.4.6 with libstdc++ 6.0.3. All of my development is being
 > done on a dual AMD Opteron machine running Solaris 10. I have
 > attempted to upgrade my compiler several times over the past few
 > years but have always had the same problem, which is application
 > hanging and driving CPU utilization up to 100%. This was impossible
 > to debug on a single processor machine, but now binding my
 > application to one processor on a dual-processor machine allows the
 > machine to remain responsive when the app goes to 100% on one CPU
 > giving me an opportunity to debug.
 > 
 > I am now determined to find the cause of this hanging with 100%
 > CPU. It is obvious from stack traces that several threads are hung
 > inside of either __gnu_cxx::__exchange_and_add () or
 > __gnu_cxx::__atomic_add (), and I suspect these functions in
 > dead-lock are responsible for driving CPU utilization to
 > 100%. Unfortunately, so far I am unable to find any reason that
 > these would or could dead-lock.

Well, you should be able to step through these atomic functions and
see what they're doing: as they're at 100% cpu they aren't waiting on
mutexes.  So, attach your trusty debugger to the threads and
single-step to see what's going on.

 > Most often these functions are being called from a
 > std::basic_string constructor or destructor, but I am also seeing
 > it on occasion inside of a std::locale constructor or destructor
 > within std::basic_stringstream constructor or destructor. Perhaps
 > related to reference counting? I am not certain, but I suspect that
 > the cause of this is not necessarily with the particular
 > std::string or std::stringstream instances that are showing up
 > repeatedly in my stack traces as I have exhaustively checked and
 > re-checked and they seem completely kosher. This leads me to
 > believe that it is a bug elsewhere in my code (or in an included
 > library?) that is indirectly causing these problems, and this is
 > what has brought me to this list.

Valgrind would almost certainly help you here, but Solaris isn't
supported.  Purify does support Solaris, but costs real money.

Andrew.