RE: libstdc++ debugging, was:Re: Heads up: Several thousand g++ and libstdc++ test failures on sparc[64]

[Pétur Runólfsson <peturr02 at ru dot is>]

Andreas Tobler wrote:
> Ok, I did some debugging on this topic. But I don't know if I do it
> correct. Could anyone please tell me if it is the right way?
> I attach my dbg log.
> At the moment I'm stuck since I don't want to follow the wrong
> direction, if.
> Any corrections, confirmations are appreciated.

> section .text._Z12__atomic_addPVii
> _Z12__atomic_addPVii()

OK.

> Hm, I see _Z12__atomic_addPVii() several times in the disassembled
> libstdc++.
> Strange!
> 
> Seven times in total. Why?

It's a static function defined in a header, so there's one copy in
each object file that uses it.

> pstack core:
> 
> core 'core' of 14087:   ./header_cassert.exe
>  ff2cc4c0 _Z12__atomic_addPVii (0, 1, 0, ff35b3d4, 0, 20cb0) + 2c
>  ff2e36e4 _ZNSt8ios_base4InitC1Ev (ff35ba54, 0, 0, 0, 0, 400) + 84
>  ff2e1714 _Z41__static_initialization_and_destruction_0ii (1, 1, 0, 1, 0, 1) + 60

So it seems that the crash occurs in a call to __atomic_add that is
called from ios_base::Init::Init().

A search for __atomic_add turns up 5 files:

include/bits/basic_string.h
include/bits/ios_base.h
include/bits/locale_classes.h
include/ext/pool_allocator.h
config/*/atomicity.h

Nothing called from ios_base::Init::Init() should ever use basic_string
or __pool_alloc. The use in ios_base.h is in
_Callback_list::_M_add_reference. Nothing in the library ever registers
callbacks.

This leaves locale_classes.h. There are two uses of __atomic_add, in
locale::facet::_M_add_reference and locale::_Impl::_M_add_reference.

_M_add_reference is called from 11 locations in the source. Of these
the ones in locale::_Impl::_M_install_cache,
locale::_Impl::_M_install_facet,
locale::_Impl::_Impl(const _Impl&, size_t), locale::global,
locale::operator= and locale::locale(const char*) should never be
called from ios_base::Init::Init(). locale::locale(const locale&)
will only be called if another locale has been successfully
constructed before. This leaves locale::locale():

  locale::locale() throw()
  { 
    _S_initialize(); 
    (_M_impl = _S_global)->_M_add_reference(); 
  }

The most likely reason for _M_add_reference to crash is if the this
pointer (_S_global) is NULL. It should be easy to verify this.

If the above theory is correct, the problem is in
locale::_S_initialize:

  void
  locale::_S_initialize_once()
  {
    // 2 references.
    // One reference for _S_classic, one for _S_global
    _S_classic = new (&c_locale_impl) _Impl(2);
    _S_global = _S_classic; 	    
    new (&c_locale) locale(_S_classic);
  }

  void  
  locale::_S_initialize()
  {
#ifdef __GTHREADS
    if (__gthread_active_p())
      __gthread_once(&_S_once, _S_initialize_once);
    else
#endif
      {
	if (!_S_classic)
	  _S_initialize_once();
      }
  }

We now have two possibilities:
1) _S_initialize_once is not called.
2) _S_initialize_once is called, but it fails to initialize _S_global,
or _S_global is clobbered sometime after it is initialized.

Can you find out which?
Does removing the #ifdef __GHTREADS block help? If it does, feel
free to post a patch. It will make locale initialization
non-threadsafe, but that's how things were before this mess anyway.

Regards,
Petur