do concurrent in parallel? speed-up?

[Tobias Burnus]

Hello!

Tim Prince wrote:
> On 2/11/2014 11:34 AM, Anton Shterenlikht wrote:
>> How can I get speed-up from using
>> DO CONCURRENT with gfortran 4.8/4.9?

In terms of the Fortran standard, the user just guarantees that the look 
doesn't have some hidden dependencies on the loop index order. (And 
provides some constraints which permit the compiler to catch some 
obvious cases.) A compiler can use this extra bits to do additional 
optimizations, including parallelization (vectorization, thread 
parallelization, ...).

In case of gfortran 4.8, it is simply converted into a normal loop - 
although with masks it gets a bit more complicated.

In case of gfortran 4.9, a DO CONCURRENT loop is additionally annotated; 
in C/C++ that annotation is "#pragma ivdep": 
http://gcc.gnu.org/onlinedocs/gcc/Loop-Specific-Pragmas.html – see 
example in the documentation. It does:

"With this pragma, the programmer asserts that there are no loop-carried 
dependencies which would prevent that consecutive iterations of the 
following loop can be executed concurrently with SIMD (single 
instruction multiple data) instructions."

Except for Fortran's variables with pointer or target attribute, the 
alias rules often permit the compiler to perform the vectorization (= 
SIMD concurrency) without this extra bit of information.

These extra bits of information can also help with autoparallelization, 
but autoparallelization never really worked for me.


Regarding thread parallelization: In principle, the compiler could 
automatically add a "!$omp parallel do" to those do-concurrent loops. 
However, that has a lot of caveats: Does the user have his own threads 
and doesn't like threads for all or some do-concurrent loops? If is 
profitable to spawn new threads only for that loop?

Thus, while someone might add a flag to use OpenMP's (libgomp; parallel 
do) or Cilk Plus's (libcilkrts; cilk_for) automatically, I think some 
hand-tuned parallelization is usually better.

Despite all the problems of being not integrated into the language and 
possibly not having the cleanest concurrency model, OpenMP seems to be 
the best way to do thread parallelization of Fortran programs and with 
OpenMP 4's "omp simd" one can also better control SIMD for loops 
(including reductions) - or one can even use OpenMP (or OpenACC) to do 
parallelization via offloading to an accelerator.

[Side note: OpenMP 4's new directives are not yet supported by gfortran 
4.9 only by gcc/g++ 4.9. Additionally, OpenMP/OpenACC offloading is not 
yet implemented - but it is currently developed and will presumably 
supported in the next version (5.0?).]


>> Are there any special compiler flags?

Only the general -ftree-parallelize-loops=n. Or going the route to 
OpenMP parallelization. If there is enough interest, one can also think 
of the automatic "parallel do"/cilk_for" parallelization via some 
-fdo-concurrent=... flag. But I am not sure whether it really makes sense.

* * *

> I've turned my masked do concurrents into DO loops with MERGE for the 
> benefit of gfortran 4.9. ifort needs VECTOR ALIGNED directives to get 
> full performance of masked DO CONCURRENT, and it's still fully 
> effective only for single thread vectorization.

I wonder whether we need to add some alignment support also to gfortran. 
In C/C++, one can use _mm_malloc/posix_memalign, 
__builtin_assume_aligned (and __attribute__((aligned))) - although there 
is not much progress in the standard front [C++1x has alignof/alignas].

However, for Fortran, there doesn't seem to be an agreed way to get 
aligned memory. Intel seems to have some directive, which applies both 
static and dynamically allocated memory (i.e. it influences a later 
ALLOCATE). I wonder whether gfortran should follow.

Tobias