Bug 65467 - [libgomp] sorry, unimplemented: '_Atomic' with OpenMP
Summary: [libgomp] sorry, unimplemented: '_Atomic' with OpenMP
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: c (show other bugs)
Version: 5.0
: P3 enhancement
Target Milestone: ---
Assignee: Jakub Jelinek
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-03-19 09:01 UTC by Sebastian Huber
Modified: 2016-09-16 07:21 UTC (History)
2 users (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed: 2015-12-28 00:00:00


Attachments
gcc7-pr65467-wip.patch (5.11 KB, patch)
2016-08-31 16:17 UTC, Jakub Jelinek
Details | Diff
gcc7-pr65467.patch (7.72 KB, patch)
2016-09-02 11:15 UTC, Jakub Jelinek
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Sebastian Huber 2015-03-19 09:01:34 UTC
It seems that <stdatomic.h> is not available with -fopenmp:

stdatomic.h:40:1: sorry, unimplemented: '_Atomic' with OpenMP
 typedef _Atomic _Bool atomic_bool;

Is this a principal problem with the OpenMP standard or libgomp?

The __atomic built-ins seem to work, e.g.

int f(int *a, int b)
{
  return __atomic_fetch_add(a, b, 0);
}
Comment 1 Jakub Jelinek 2015-03-19 09:58:57 UTC
This is indeed just a big hammer approach.
The OpenMP standard only supports C up to C99 and C++ up to C++98 at this point, for _Atomic it is non-trivial to figure out how it should behave with different clauses etc.  But indeed, it would be better to just complain only if _Atomic is somehow used in OpenMP regions, but that would require first writing testcases for all the different possibilities where _Atomic could appear.
Comment 2 jsm-csl@polyomino.org.uk 2015-03-19 23:33:50 UTC
The issue is that someone needs to go through all the parsing for OpenMP 
constructs, and figure out exactly where to add calls to 
convert_lvalue_to_rvalue (if an OpenMP construct reads the value of an 
object, reading the value of an _Atomic object must be an atomic load) and 
what other special handling might be needed (if an OpenMP construct writes 
to an object, it must be an atomic store; if it both reads and writes, 
some form of compare-and-exchange may be needed).
Comment 3 Jeff Hammond 2016-08-30 16:11:32 UTC
This is awful.  How do I disable this horrible thing?

I am using OpenMP to create a thread pool, because C11 threads are still not implemented in glibc, and all of my access to C11 _Atomic variables use C11 atomic operations, so my code is correct.  Do you seriously pick this one time to prevent the user from even trying to write incorrect code, while allowing an uncountable number of others?

One of the motivations for writing code that mixes C11 and OpenMP is because I am a member of the OpenMP working group devoted to supporting C11 and C++14 in the OpenMP standard.  By refusing to allow me to experiment with OpenMP+C11, you actively harm progress in the OpenMP standard that would allow you to resolve the semantic ambiguity that motivated disabling C11+OpenMP in the first place.
Comment 4 Jeff Hammond 2016-08-30 18:00:38 UTC
Apparently, the GCC team wants to make it impossible for anyone to build software where independent components that share CFLAGS in the build system cannot use both the C11 atomics header and the OpenMP flag.  It doesn't matter if you use either feature, literally including stdatomic.h and compiling with -fopenmp is impossible.

So projects that use autotools and support CFLAGS=-fopenmp now need to segregate the build system to compile any source files that include stdatomic.h using a different set of options from the default?

It is really hard to imagine how someone came to the conclusion this was a reasonable thing to do.

Anyways, here is the trivial test program that is broken by CFLAGS=-fopenmp.

#include <stdatomic.h>

int main(void)
{
    return 0;
}
Comment 5 Andrew Pinski 2016-08-30 18:08:32 UTC
From the original discussions on why this is disabled:
_Atomic support is currently disabled for Objective-C and OpenMP.  For
both (but mainly OpenMP), the relevant parser code needs checking to
determine where convert_lvalue_to_rvalue calls need inserting to
ensure that accesses to atomic variables involve atomic loads.  For
Objective-C, there are also various special cases of compound
assignment that need special handling for atomics just as standard C
compound assignment is handled differently for atomics, as well as
some TYPE_MAIN_VARIANT calls to check for correctness for atomics; see
the comment on the relevant sorry () call for details.  OpenMP should
also have TYPE_MAIN_VARIANT uses checked as well as a use of
TYPE_QUALS_NO_ADDR_SPACE for a diagnostic in
c_parser_omp_declare_reduction (where the diagnostic refers to a
particular list of qualifiers).

So it looks like there is more than even what Jakub listed.

Also patches are welcome to handle OpenMP and _Atomic better.
Comment 6 Andrew Pinski 2016-08-30 18:11:02 UTC
(In reply to Jeff Hammond from comment #3)
> Do you seriously pick this one time to prevent the user from even trying to
> write incorrect code, while allowing an uncountable number of others?

This is different because the semantics are not defined at all.

> 
> One of the motivations for writing code that mixes C11 and OpenMP is because
> I am a member of the OpenMP working group devoted to supporting C11 and
> C++14 in the OpenMP standard.  By refusing to allow me to experiment with
> OpenMP+C11, you actively harm progress in the OpenMP standard that would
> allow you to resolve the semantic ambiguity that motivated disabling
> C11+OpenMP in the first place.

If you are part of the working group then you should be able to help define the semantics instead of complaining we disable things :).
Comment 7 Jeff Hammond 2016-08-30 18:17:31 UTC
The fact that the parser doesn't handle a particle case where something might go wrong is no reason to have the compiler refuse to compile code that includes stdatomic.h with -fopenmp.  Look at my example and tell me what possible thing can go wrong in it that justifies aborting the compilation.

This sort of attack on user experience is unprecedented in my career of programming.  Do you break every other mixture of programming standard semantics that is currently undefined?  I can think of others that GCC allows, but will not list them, out of fear that someone will decide to break those as well.

At most, this should have been a warning to indicate to the user that OpenMP constructs do not correctly interact with _Atomic and the user should take care to rely on only what is supported by ISO C11 and OpenMP 4.5.
Comment 8 Jeff Hammond 2016-08-30 18:21:25 UTC
(In reply to Andrew Pinski from comment #6)
> (In reply to Jeff Hammond from comment #3)
> > Do you seriously pick this one time to prevent the user from even trying to
> > write incorrect code, while allowing an uncountable number of others?
> 
> This is different because the semantics are not defined at all.

So GCC refuses to compile any code that potentially includes undefined behavior?

Please tell me about the undefined behavior in the following program, when compiled with -fopenmp:

#include <stdatomic.h>

int main(void)
{
    return 0;
}

> > One of the motivations for writing code that mixes C11 and OpenMP is because
> > I am a member of the OpenMP working group devoted to supporting C11 and
> > C++14 in the OpenMP standard.  By refusing to allow me to experiment with
> > OpenMP+C11, you actively harm progress in the OpenMP standard that would
> > allow you to resolve the semantic ambiguity that motivated disabling
> > C11+OpenMP in the first place.
> 
> If you are part of the working group then you should be able to help define
> the semantics instead of complaining we disable things :).

As I said already, I am trying to define them, but that has nothing to do with the fact that GCC unnecessary broke an infinite number of valid programs.
Comment 9 Andrew Pinski 2016-08-30 18:25:11 UTC
(In reply to Jeff Hammond from comment #8)
> So GCC refuses to compile any code that potentially includes undefined
> behavior?

Semantics not being defined is different than undefined behavior.
Comment 10 Jeff Hammond 2016-08-30 18:30:55 UTC
(In reply to Andrew Pinski from comment #9)
> (In reply to Jeff Hammond from comment #8)
> > So GCC refuses to compile any code that potentially includes undefined
> > behavior?
> 
> Semantics not being defined is different than undefined behavior.

GCC happily compiles a C++11 OpenMP program that is equivalent to the C11 OpenMP program that it will not compile.

GCC happily compiles the following Fortran 2008 OpenMP program that actually does something that could be considered undefined.

$ gfortran-6 -fopenmp -std=f2008 -fcoarray=single caf-openmp.f 
$ cat caf-openmp.f 
      program atomic
      use iso_fortran_env
      use omp_lib
      implicit none
      integer :: i
      integer(atomic_int_kind) :: atom[*]
      call atomic_define (atom[1], this_image())
      !$OMP ATOMIC
      atom[1] = -this_image()
      end program atomic

If you want to break user experience for OpenMP programmers, please do it systematically.
Comment 11 Jakub Jelinek 2016-08-31 16:17:18 UTC
Created attachment 39524 [details]
gcc7-pr65467-wip.patch

Untested WIP patch.  This attempts to handle _Atomic qualified vars/expressions etc. where it is easy and non-controversial, and error out otherwise.
Testsuite coverage for the rejections is still missing.
The cases I plan to (in the current patch or later) reject are:
1) omp loop iterators (for, simd, distribute, taskloop) - I think it makes little sense to support that, and would be a nightmare to support
2) _Atomic vars in linear clause (again, little sense, not that hard to support, but right now looks like wasted time to do until omp-lang decides)
3) _Atomic vars in reduction clause and _Atomic types in omp declare reduction (again, little sense to do that, and quite a lot of work to support)
4) _Atomic vars in aligned clause (it is fine if it is in alignment expression)
5) _Atomic x expression in #pragma omp atomic (v or expr can be _Atomic)
6) _Atomic return type on omp declare simd functions, or _Atomic arguments unless they are uniform (again, not impossible to handle, but lots of work and little sense) - this isn't rejected, but just warned and declare simd ignored
7) _Atomic vars in explicit map/to/from clauses (this a nightmare to support, right now the runtime library doesn't have support to run special functions to copy the data to/from, it is similar to not actually running ctors/dtors, but just doing memcpy-ish transfers; and it isn't just the host <-> device transfers, but also possible copying into a temporary for target nowait
Not in the patch yet:
8) reject implicit map clauses for _Atomic vars
9) reject firstprivate on _Atomic vars on the target construct (not as hard as map, but also very complicated)
Comment 12 Jakub Jelinek 2016-09-01 13:58:26 UTC
Actually, firstprivate on _Atomic vars in target construct could be implemented just by forcing it into a temporary with non-_Atomic qualified type on the host side (i.e. atomically loading it), firstprivatizing such temporary instead of the original _Atomic variable, privatizing the _Atomic variable instead and then assigning the privatized var the temporary's firstprivatized copy in the target code.  Not going to implement it unless it gets standardized though.
Comment 13 Jakub Jelinek 2016-09-02 11:15:00 UTC
Created attachment 39540 [details]
gcc7-pr65467.patch

Untested updated patch I'm going to bootstrap/regtest.
Comment 14 Jakub Jelinek 2016-09-02 18:38:39 UTC
Author: jakub
Date: Fri Sep  2 18:38:07 2016
New Revision: 239964

URL: https://gcc.gnu.org/viewcvs?rev=239964&root=gcc&view=rev
Log:
	PR c/65467
	* gimplify.c (gimplify_adjust_omp_clauses_1): Diagnose implicit
	map and firstprivate clauses on target construct for _Atomic
	qualified decls.
	(gimplify_adjust_omp_clauses): Diagnose explicit firstprivate clauses
	on target construct for _Atomic qualified decls.
	* omp-low.c (use_pointer_for_field): Return true for _Atomic qualified
	decls.
	* omp-simd-clone.c (simd_clone_clauses_extract): Warn and give up for
	_Atomic qualified arguments not mentioned in uniform clause.
c/
	* c-parser.c (c_parser_declspecs): Don't sorry about _Atomic if
	flag_openmp.
	(c_parser_omp_variable_list): Use convert_lvalue_to_rvalue
	instead of mark_exp_read on low_bound/length expression.
	(c_parser_omp_clause_num_gangs, c_parser_omp_clause_num_threads,
	c_parser_omp_clause_num_tasks, c_parser_omp_clause_grainsize,
	c_parser_omp_clause_priority, c_parser_omp_clause_hint,
	c_parser_omp_clause_num_workers, c_parser_oacc_shape_clause,
	c_parser_oacc_clause_tile, c_parser_omp_clause_schedule,
	c_parser_omp_clause_vector_length, c_parser_omp_clause_num_teams,
	c_parser_omp_clause_thread_limit, c_parser_omp_clause_aligned,
	c_parser_omp_clause_linear, c_parser_omp_clause_safelen,
	c_parser_omp_clause_simdlen, c_parser_omp_clause_device,
	c_parser_omp_clause_dist_schedule): Use convert_lvalue_to_rvalue
	instead of mark_expr_read.
	(c_parser_omp_declare_reduction): Reject _Atomic qualified types.
	* c-objc-common.h (LANG_HOOKS_OMP_CLAUSE_COPY_CTOR,
	LANG_HOOKS_OMP_CLAUSE_ASSIGN_OP): Redefine.
	* c-tree.h (c_omp_clause_copy_ctor): New prototype.
	* c-typeck.c (handle_omp_array_sections_1): Diagnose _Atomic qualified
	array section bases outside of depend clause, for depend clause
	use convert_lvalue_to_rvalue on the base.
	(c_finish_omp_clauses): Reject _Atomic qualified vars in reduction,
	linear, aligned, map, to and from clauses.
	(c_omp_clause_copy_ctor): New function.
c-family/
	* c-omp.c (c_finish_omp_atomic): Reject _Atomic qualified expressions.
	(c_finish_omp_for): Reject _Atomic qualified iterators.
testsuite/
	* gcc.dg/gomp/_Atomic-1.c: New test.
	* gcc.dg/gomp/_Atomic-2.c: New test.
	* gcc.dg/gomp/_Atomic-3.c: New test.
	* gcc.dg/gomp/_Atomic-4.c: New test.
	* gcc.dg/gomp/_Atomic-5.c: New test.

Added:
    trunk/gcc/testsuite/gcc.dg/gomp/_Atomic-1.c
    trunk/gcc/testsuite/gcc.dg/gomp/_Atomic-2.c
    trunk/gcc/testsuite/gcc.dg/gomp/_Atomic-3.c
    trunk/gcc/testsuite/gcc.dg/gomp/_Atomic-4.c
    trunk/gcc/testsuite/gcc.dg/gomp/_Atomic-5.c
Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/c-family/ChangeLog
    trunk/gcc/c-family/c-omp.c
    trunk/gcc/c/ChangeLog
    trunk/gcc/c/c-objc-common.h
    trunk/gcc/c/c-parser.c
    trunk/gcc/c/c-tree.h
    trunk/gcc/c/c-typeck.c
    trunk/gcc/gimplify.c
    trunk/gcc/omp-low.c
    trunk/gcc/omp-simd-clone.c
    trunk/gcc/testsuite/ChangeLog
Comment 15 Andreas Schwab 2016-09-03 08:18:35 UTC
FAIL: gcc.dg/gomp/_Atomic-4.c  (test for warnings, line 7)
Comment 16 Jakub Jelinek 2016-09-03 08:31:25 UTC
(In reply to Andreas Schwab from comment #15)
> FAIL: gcc.dg/gomp/_Atomic-4.c  (test for warnings, line 7)

Does

--- gcc/testsuite/gcc.dg/gomp/_Atomic-4.c.jj	2016-09-02 20:36:22.000000000 +0200
+++ gcc/testsuite/gcc.dg/gomp/_Atomic-4.c	2016-09-03 10:30:29.708581112 +0200
@@ -1,6 +1,7 @@
 /* PR c/65467 */
 /* { dg-do compile } */
 /* { dg-additional-options "-std=c11" } */
+/* { dg-require-effective-target vect_simd_clones } */
 
 #pragma omp declare simd
 int

fix it?
Comment 17 Andreas Schwab 2016-09-03 09:11:38 UTC
FAIL -> UNSUPPORTED
Comment 18 Jakub Jelinek 2016-09-03 09:18:43 UTC
(In reply to Andreas Schwab from comment #17)
> FAIL -> UNSUPPORTED

That is expected on targets that don't provide compute_vecsize_and_simdlen target hook.  If it is a target with reasonable vector support (not really counting here ia64 or alpha, but e.g. powerpc*, s390*, aarch64, arm etc. should),
then declare simd is not creating any simd clones.
I'll commit the change.
Comment 19 Jakub Jelinek 2016-09-03 09:20:36 UTC
Author: jakub
Date: Sat Sep  3 09:20:03 2016
New Revision: 239970

URL: https://gcc.gnu.org/viewcvs?rev=239970&root=gcc&view=rev
Log:
	PR c/65467
	* gcc.dg/gomp/_Atomic-4.c: Require vect_simd_clones effective target.

Modified:
    trunk/gcc/testsuite/ChangeLog
    trunk/gcc/testsuite/gcc.dg/gomp/_Atomic-4.c
Comment 20 Andreas Schwab 2016-09-03 09:21:08 UTC
aarch64 also fails.
Comment 21 Jeff Hammond 2016-09-03 16:46:01 UTC
Thanks.  This is great.  I built GCC master last night and can now compile both the trivial test program and a more interesting one that encapsulates what I actually need to work to make progress on OpenMP 5 and other activities.

/* trivial */

#include <stdatomic.h>

int main(void)
{
    return 0;
}

/* nontrivial */

#if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 201112L) && !defined(__STDC_NO_ATOMICS__)

#include <stdio.h>
#include <stdlib.h>
#include <stdbool.h>

#include <stdatomic.h>

#ifdef _OPENMP
# include <omp.h>
#else
# error No OpenMP support!
#endif

#ifdef SEQUENTIAL_CONSISTENCY
int load_model  = memory_order_seq_cst;
int store_model = memory_order_seq_cst;
#else
int load_model  = memory_order_acquire;
int store_model = memory_order_release;
#endif

int main(int argc, char * argv[])
{
    int nt = omp_get_max_threads();
#if 1
    if (nt != 2) omp_set_num_threads(2);
#else
    if (nt < 2)      omp_set_num_threads(2);
    if (nt % 2 != 0) omp_set_num_threads(nt-1);
#endif

    int iterations = (argc>1) ? atoi(argv[1]) : 100;

    printf("thread ping-pong benchmark\n");
    printf("num threads  = %d\n", omp_get_max_threads());
    printf("iterations   = %d\n", iterations);
#ifdef SEQUENTIAL_CONSISTENCY
    printf("memory model = %s\n", "seq_cst");
#else
    printf("memory model = %s\n", "acq-rel");
#endif
    fflush(stdout);

    _Atomic int left_ready  = -1;
    _Atomic int right_ready = -1;

    int left_payload  = 0;
    int right_payload = 0;

    #pragma omp parallel
    {
        int me      = omp_get_thread_num();
        /// 0=left 1=right
        bool parity = (me % 2 == 0);

        int junk = 0;

        /// START TIME
        #pragma omp barrier
        double t0 = omp_get_wtime();

        for (int i=0; i<iterations; ++i) {

            if (parity) {

                /// send to left
                left_payload = i;
                atomic_store_explicit( &left_ready, i, store_model);

                /// recv from right
                while (i != atomic_load_explicit( &right_ready, load_model));
                //printf("%d: left received %d\n", i, right_payload);
                junk += right_payload;

            } else {

                /// recv from left
                while (i != atomic_load_explicit( &left_ready, load_model));
                //printf("%d: right received %d\n", i, left_payload);
                junk += left_payload;

                ///send to right
                right_payload = i;
                atomic_store_explicit( &right_ready, i, store_model);

            }

        }

        /// STOP TIME
        #pragma omp barrier
        double t1 = omp_get_wtime();

        /// PRINT TIME
        double dt = t1-t0;
        #pragma omp critical
        {
            printf("total time elapsed = %e\n", dt);
            printf("time per iteration = %e\n", dt/iterations);
            printf("%d\n", junk);
        }
    }

    return 0;
}

#else  // C11
#error You need C11 atomics for this test!
#endif // C11
Comment 22 Jakub Jelinek 2016-09-16 07:21:04 UTC
Fixed.