Bug 100164 - [11/12 Regression] semaphore_impl not declared on AIX
Summary: [11/12 Regression] semaphore_impl not declared on AIX
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: libstdc++ (show other bugs)
Version: 11.0
: P1 blocker
Target Milestone: 11.0
Assignee: Thomas Rodgers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-04-20 21:42 UTC by David Edelsohn
Modified: 2021-04-21 16:05 UTC (History)
6 users (show)

See Also:
Host:
Target: powerpc-ibm-aix*
Build:
Known to work:
Known to fail:
Last reconfirmed: 2021-04-20 00:00:00


Attachments
Disable <semaphore> on unsupported targets (775 bytes, application/mbox)
2021-04-20 22:44 UTC, Thomas Rodgers
Details
pre-processed source for 17_intro/headers/c++2020/all_attributes.cc (639.26 KB, text/plain)
2021-04-20 23:07 UTC, David Edelsohn
Details
Disable <semaphore> on unsupported targets (1.47 KB, application/mbox)
2021-04-20 23:15 UTC, Thomas Rodgers
Details
Work around broken macro name (1.44 KB, application/mbox)
2021-04-21 05:09 UTC, Thomas Rodgers
Details

Note You need to log in before you can comment on or make changes to this bug.
Description David Edelsohn 2021-04-20 21:42:08 UTC
The last-minute change to libstdc++ at the time of the GCC 11 branch has caused severe regressions on AIX.

All of the testsuite regressions are of the form:

/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/bits/semaphore_base.h:259: 
error: #error "No suitable semaphore implementation available"
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:43: error: '__semaphore_impl' has not been declared
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:47: error: '__semaphore_impl' has not been declared
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:49: error: '__semaphore_impl' does not name a type
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:66: error: '_M_sem' was not declared in this scope
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:70: error: '_M_sem' was not declared in this scope
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:74: error: '_M_sem' was not declared in this scope
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:53: error: class 
'std::counting_semaphore<__least_max_value>' does not have any field named '_M_sem'
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:67: error: '_M_sem' was not declared in this scope
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:71: error: '_M_sem' was not declared in this scope
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:75: error: '_M_sem' was not declared in this scope
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:80: error: '_M_sem' was not declared in this scope
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:85: error: '_M_sem' was not declared in this scope
Comment 1 David Edelsohn 2021-04-20 21:44:12 UTC
Confirmed
Comment 2 Thomas Rodgers 2021-04-20 22:02:59 UTC
Investigating
Comment 3 Thomas Rodgers 2021-04-20 22:44:13 UTC
Created attachment 50643 [details]
Disable <semaphore> on unsupported targets

This patch is probably not the most elegant way to do this, it probably should be a dg-effective-target check and it disables <semaphore> on platforms where there is a suitable Posix implementation, but this feature is experimental for GCC11.
Comment 4 Thomas Rodgers 2021-04-20 22:48:58 UTC
It would appear that I cannot log into either of the AIX machines in the compile farm.
Comment 5 Jakub Jelinek 2021-04-20 23:04:59 UTC
Comment on attachment 50643 [details]
Disable <semaphore> on unsupported targets

+#else
+      __gthread_yield();

The changelog said you want to use __thread_yield(); there instead...
Comment 6 David Edelsohn 2021-04-20 23:07:33 UTC
Created attachment 50644 [details]
pre-processed source for 17_intro/headers/c++2020/all_attributes.cc
Comment 7 Thomas Rodgers 2021-04-20 23:15:29 UTC
Created attachment 50645 [details]
Disable <semaphore> on unsupported targets

Let's try this with the right patch attached this time
Comment 8 David Edelsohn 2021-04-20 23:51:46 UTC
I am not certain why you cannot log in to the compile farm system.
I am testing the patch on one of the AIX systems inside IBM.
Comment 9 David Edelsohn 2021-04-21 00:45:21 UTC
The previous semaphore_base.h implementation had a fallback that hid a bug in the macros:

#if defined _GLIBCXX_HAVE_LINUX_FUTEX && !_GLIBCXX_REQUIRE_POSIX_SEMAPHORE
  // Use futex if available and didn't force use of POSIX
  using __fast_semaphore = __atomic_semaphore<__detail::__platform_wait_t>;
#elif _GLIBCXX_HAVE_POSIX_SEMAPHORE
  using __fast_semaphore = __platform_semaphore;
#else
  using __fast_semaphore = __atomic_semaphore<ptrdiff_t>;
#endif

The problem is that libstdc++ configure defines
_GLIBCXX_HAVE_POSIX_SEMAPHORE in config.h.  libstdc++ uses sed to
rewrite config.h to c++config.h and prepends _GLIBCXX_, so c++config.h
contains

#define _GLIBCXX__GLIBCXX_HAVE_POSIX_SEMAPHORE 1

And bits/semaphore_base.h is not testing that corrupted macro.  Either
semaphore_base.h needs to test for the corrupted macro, or libtsdc++
configure needs to define HAVE_POSIX_SEMAPHORE without itself
prepending _GLIBCXX_  so that the c++config.h rewriting works
correctly and defines the correct macro for semaphore_base.h.
Comment 10 Thomas Rodgers 2021-04-21 05:09:29 UTC
Created attachment 50646 [details]
Work around broken macro name

This patch works around the borked macro name that David pointed out on the mailing list. I left in the commented out _GLIBCXX_HAVE_POSIX_SEMAPHORE checks and have temporarily replaced them with _GLIBCXX__GLIBCXX_HAVE_POSIX_SEMAPHORE, and forced the posix semaphore test to always run if that macro is defined.

I am not sufficiently versed in the arcane ways in which config.h is transformed to c++config.h but the borked macro transformation ideally should be fixed and the commented out checks restored.
Comment 11 GCC Commits 2021-04-21 09:14:29 UTC
The master branch has been updated by Jakub Jelinek <jakub@gcc.gnu.org>:

https://gcc.gnu.org/g:2d4c3af94f84c874cfddc753dc0f34ebf7fc11d9

commit r12-22-g2d4c3af94f84c874cfddc753dc0f34ebf7fc11d9
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Wed Apr 21 11:09:25 2021 +0200

    Fix AIX libstdc++ semaphore support [PR100164]
    
    > > The #error would not be hit if _GLIBCXX_HAVE_POSIX_SEMAPHORE were defined,
    > > but it shows up in your error report.
    
    > You now have pinpointed the problem.
    
    > It's not that AIX doesn't have semaphore, but that the code previously
    > had a fallback that hid a bug in the macros:
    
      // Use futex if available and didn't force use of POSIX
      using __fast_semaphore = __atomic_semaphore<__detail::__platform_wait_t>;
      using __fast_semaphore = __platform_semaphore;
      using __fast_semaphore = __atomic_semaphore<ptrdiff_t>;
    
    > The problem is that libstdc++ configure defines
    > _GLIBCXX_HAVE_POSIX_SEMAPHORE in config.h.  libstdc++ uses sed to
    > rewrite config.h to c++config.h and prepends _GLIBCXX_, so c++config.h
    > contains
    
    > And bits/semaphore_base.h is not testing that corrupted macro.  Either
    > semaphore_base.h needs to test for the corrupted macro, or libtsdc++
    > configure needs to define HAVE_POSIX_SEMAPHORE without itself
    > prepending _GLIBCXX_  so that the c++config.h rewriting works
    > correctly and defines the correct macro for semaphore_base.h.
    
    The include/Makefile.am sed is:
            sed -e 's/HAVE_/_GLIBCXX_HAVE_/g' \
                -e 's/PACKAGE/_GLIBCXX_PACKAGE/g' \
                -e 's/VERSION/_GLIBCXX_VERSION/g' \
                -e 's/WORDS_/_GLIBCXX_WORDS_/g' \
                -e 's/_DARWIN_USE_64_BIT_INODE/_GLIBCXX_DARWIN_USE_64_BIT_INODE/g' \
                -e 's/_FILE_OFFSET_BITS/_GLIBCXX_FILE_OFFSET_BITS/g' \
                -e 's/_LARGE_FILES/_GLIBCXX_LARGE_FILES/g' \
                -e 's/ICONV_CONST/_GLIBCXX_ICONV_CONST/g' \
                -e '/[       ]_GLIBCXX_LONG_DOUBLE_COMPAT[   ]/d' \
                -e '/[       ]_GLIBCXX_LONG_DOUBLE_ALT128_COMPAT[    ]/d' \
                < ${CONFIG_HEADER} >> $@ ;\
    so for many macros one needs _GLIBCXX_ prefixes already in configure,
    as can be seen in grep AC_DEFINE.*_GLIBCXX configure.ac acinclude.m4
    But _GLIBCXX_HAVE_POSIX_SEMAPHORE is the only one that shouldn't have
    that prefix because the sed is adding that.
    E.g. on i686-linux, I see
    grep _GLIBCXX__GLIBCXX c++config.h
    that proves it is the only broken one.
    
    So this change fixes the acinclude.m4 side.
    
    2021-04-21  Jakub Jelinek  <jakub@redhat.com>
    
            PR libstdc++/100164
            * acinclude.m4: For POSIX semaphores AC_DEFINE HAVE_POSIX_SEMAPHORE
            rather than _GLIBCXX_HAVE_POSIX_SEMAPHORE.
            * configure: Regenerated.
            * config.h.in: Regenerated.
Comment 12 Jonathan Wakely 2021-04-21 09:15:18 UTC
(In reply to Jakub Jelinek from comment #5)
> The changelog said you want to use __thread_yield(); there instead...

Yes, that was fixed in r11-7293 (the patch attached in comment 3 was r11-7248).
Comment 13 GCC Commits 2021-04-21 09:16:03 UTC
The releases/gcc-11 branch has been updated by Jakub Jelinek <jakub@gcc.gnu.org>:

https://gcc.gnu.org/g:0857fc3820bee04771f60a86b9dd5627b3ea1904

commit r11-8270-g0857fc3820bee04771f60a86b9dd5627b3ea1904
Author: Jakub Jelinek <jakub@redhat.com>
Date:   Wed Apr 21 11:09:25 2021 +0200

    Fix AIX libstdc++ semaphore support [PR100164]
    
    > > The #error would not be hit if _GLIBCXX_HAVE_POSIX_SEMAPHORE were defined,
    > > but it shows up in your error report.
    
    > You now have pinpointed the problem.
    
    > It's not that AIX doesn't have semaphore, but that the code previously
    > had a fallback that hid a bug in the macros:
    
      // Use futex if available and didn't force use of POSIX
      using __fast_semaphore = __atomic_semaphore<__detail::__platform_wait_t>;
      using __fast_semaphore = __platform_semaphore;
      using __fast_semaphore = __atomic_semaphore<ptrdiff_t>;
    
    > The problem is that libstdc++ configure defines
    > _GLIBCXX_HAVE_POSIX_SEMAPHORE in config.h.  libstdc++ uses sed to
    > rewrite config.h to c++config.h and prepends _GLIBCXX_, so c++config.h
    > contains
    
    > And bits/semaphore_base.h is not testing that corrupted macro.  Either
    > semaphore_base.h needs to test for the corrupted macro, or libtsdc++
    > configure needs to define HAVE_POSIX_SEMAPHORE without itself
    > prepending _GLIBCXX_  so that the c++config.h rewriting works
    > correctly and defines the correct macro for semaphore_base.h.
    
    The include/Makefile.am sed is:
            sed -e 's/HAVE_/_GLIBCXX_HAVE_/g' \
                -e 's/PACKAGE/_GLIBCXX_PACKAGE/g' \
                -e 's/VERSION/_GLIBCXX_VERSION/g' \
                -e 's/WORDS_/_GLIBCXX_WORDS_/g' \
                -e 's/_DARWIN_USE_64_BIT_INODE/_GLIBCXX_DARWIN_USE_64_BIT_INODE/g' \
                -e 's/_FILE_OFFSET_BITS/_GLIBCXX_FILE_OFFSET_BITS/g' \
                -e 's/_LARGE_FILES/_GLIBCXX_LARGE_FILES/g' \
                -e 's/ICONV_CONST/_GLIBCXX_ICONV_CONST/g' \
                -e '/[       ]_GLIBCXX_LONG_DOUBLE_COMPAT[   ]/d' \
                -e '/[       ]_GLIBCXX_LONG_DOUBLE_ALT128_COMPAT[    ]/d' \
                < ${CONFIG_HEADER} >> $@ ;\
    so for many macros one needs _GLIBCXX_ prefixes already in configure,
    as can be seen in grep AC_DEFINE.*_GLIBCXX configure.ac acinclude.m4
    But _GLIBCXX_HAVE_POSIX_SEMAPHORE is the only one that shouldn't have
    that prefix because the sed is adding that.
    E.g. on i686-linux, I see
    grep _GLIBCXX__GLIBCXX c++config.h
    that proves it is the only broken one.
    
    So this change fixes the acinclude.m4 side.
    
    2021-04-21  Jakub Jelinek  <jakub@redhat.com>
    
            PR libstdc++/100164
            * acinclude.m4: For POSIX semaphores AC_DEFINE HAVE_POSIX_SEMAPHORE
            rather than _GLIBCXX_HAVE_POSIX_SEMAPHORE.
            * configure: Regenerated.
            * config.h.in: Regenerated.
    
    (cherry picked from commit 2d4c3af94f84c874cfddc753dc0f34ebf7fc11d9)
Comment 14 David Edelsohn 2021-04-21 14:17:49 UTC
Thanks for the patch. This now removes the failure that semaphore_impl is not found.

Because the platform semaphore code never was exercised due to the bug in the macro, there are more latent bugs.

/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:74: error: 'using
 __semaphore_impl = struct std::__platform_semaphore' has no member named '_M_tr
y_acquire'; did you mean '_M_acquire'?
/tmp/GCC/powerpc-ibm-aix7.2.3.0/libstdc++-v3/include/semaphore:75: error: 'using
 __semaphore_impl = struct std::__platform_semaphore' has no member named '_M_tr
y_acquire'; did you mean '_M_acquire'?
Comment 15 Thomas Rodgers 2021-04-21 14:24:10 UTC
From the most recent patch -

+    _GLIBCXX_ALWAYS_INLINE bool
+    _M_try_acquire() noexcept
+    {
+      for (;;)
+	{
+	  auto __err = sem_trywait(&_M_semaphore);
+	  if (__err && (errno == EINTR))
+	    continue;
+	  else if (__err && (errno == EAGAIN))
+	    return false;
+	  else if (__err)
+	    std::terminate();
+	  else
+	    break;
+	}
+      return true;
+    }
+

You are correct it was never exercised. I saw then when I forced it in the test case and then added the above. I don't understand why you are seeing this error if you've applied the 0001-libstdc-Work-around-for-PR100164 patch. I don't see it locally.
Comment 16 Thomas Rodgers 2021-04-21 15:26:08 UTC
The _M_try_acquire() change should be on master and gcc-11 now.
Comment 17 Jonathan Wakely 2021-04-21 15:26:43 UTC
r12-44 and r11-8275
Comment 18 David Edelsohn 2021-04-21 16:05:37 UTC
Because the _M_try_acquire patch was not yet committed to trunk when I tested and bootstrap on trunk now is broken due to other, unrelated patches.