[PATCH v2 12/16] libstdc++: Optimise std::latch::arrive_and_wait

[Jonathan Wakely]

We don't need to wait if we know the counter has reached zero.

libstdc++-v3/ChangeLog:

	* include/std/latch (latch::arrive_and_wait): Optimise.
---
 libstdc++-v3/include/std/latch | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/libstdc++-v3/include/std/latch b/libstdc++-v3/include/std/latch
index 52a2b1489f6a..9504df0a7224 100644
--- a/libstdc++-v3/include/std/latch
+++ b/libstdc++-v3/include/std/latch
@@ -101,8 +101,21 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
     _GLIBCXX_ALWAYS_INLINE void
     arrive_and_wait(ptrdiff_t __update = 1) noexcept
     {
-      count_down(__update);
-      wait();
+      // The standard specifies this functions as count_down(update); wait();
+      // but we combine those two calls into one and avoid the wait() if we
+      // know the counter reached zero.
+
+      __glibcxx_assert(__update >= 0 && __update <= max());
+      // Use acq_rel here because an omitted wait() would have used acquire:
+      auto const __old = __atomic_impl::fetch_sub(&_M_counter, __update,
+						  memory_order::acq_rel);
+      if (std::cmp_equal(__old, __update))
+	__atomic_impl::notify_all(&_M_counter);
+      else
+	{
+	  __glibcxx_assert(std::cmp_less(__update, __old));
+	  wait();
+	}
     }
 
   private:
-- 
2.49.0