Re: [gomp] Challenges in Implementing OpenMP

[Gabriel Dos Reis <gdr at cs dot tamu dot edu>]

Sebastian Pop <sebastian.pop@cri.ensmp.fr> writes:

| On Wed, Oct 13, 2004 at 04:16:34PM -0500, Gabriel Dos Reis wrote:
| > 
| > The view here at TAMU, is slightly different.  Instead of augmenting
| > the language with CPP directives and whatnot that the programmer has
| > to manually keep track of, we prefer to realy on higher level
| > abstractions.  That is the approach taken and implemented by STAPL
| > 
| >   http://parasol.tamu.edu/compilers/research/STAPL/
| > 
| 
| After having read some papers on STAPL, I found that one of the
| targets of STAPL is OpenMP...

yes, as explained on the website too :-) 

But the point I made in my previous message is that there is a handful
number of equally good models out there and I'm convinced in the
reasons to pick that particular model over others if we feel that
the compiler needs more complexities.  And I was speaking as C++ is
concerned.

| In my opinion for transforming the code into a parallel one, you must
| gather some precious information that you don't want to just throw
| away once you go down at the level of the compiler.  Reusing this
| information in the compiler could avoid a costly computation, could
| probably be more accurate than recomputing it at a lower level, and
| finally could be useful for low level optimizations that are dependent
| on the target architecture.
| 
| Last Tuesday I had the occasion to speak with Lawrence Rauchwerger
| about STAPL. 

I'm glad you had a chance to talk to Lawrence while he is in Paris at
this moment.  I'll be in Paris in three weeks but, unfortunately, I do
not think I'll have enough time to mee Albert Cohen, you and Lawrence
simultaneously. 

| I have asked him about how the data dependence
| information is transmitted to the lower levels of the compiler.  This
| is the same problem we're facing when trying to send the data
| dependence information from GIMPLE to RTL, and I should say that this
| issue is still open.  In the case of STAPL one of the interfaces to
| the compiler is OpenMP, clearly not very useful when you know that
| STAPL computes the whole data dependence graph, but that all you can
| transmit using OpenMP directives is whether loops are parallel or
| not... 

I don't believe STAPL compute the dependency graph just to emit OpenMP
directives. 

It does other tranformations based on the semantics of the library
(pRange, vContainers, ...) that need the dependency information.  And
since that dependency information happens to around, it uses to give
directives and it usins OpenMP. 

| When you don't consider the source to source approach, ie. the

But we *do* consider the source-to-source approach which is one of the
reasons why we're working on The Pivot :-)

| STAPL compiler integrated to the code generator, the problem still
| remains difficult: translate the DDG from the "higher-end" to the
| "middle-end" then to the back-end.

Well, I think one of my points in the previous message was that
putting everyting in the compiler, because of the popular belief that
"if it is `built-in' then it is better/more efficient", is adding
enormous complexity with little or uncertain benefits.  One of the
reasons being that the compiler is going to need a huge number of
"built-in" hardcoded notions of language dependent high-level
semantics, the set of which is still moving quickly as research
advances. Looking at what GCC currently does in the simple sequential
cases with C++ programs, I believe we're far far away from seeing the
beginning of the light. That job is best left to source-to-source
transformation tools that  work in cooperation with the compiler.  The
complexity of those specific semantics is shifted to those specific
tools.

| I don't believe in the library doing all parallelization, because
| generating calls to a threading library too early is just obfuscating
| the code to be processed by the compiler.  This also is the case of
| source to source OpenMP compilers.

We can quibble on the details of exactly when the call to threading
libraries should be made, but given current transformation
technologies implemented in GCC/g++, I think that is a better approach. 

|  Instead, I think that OpenMP
| directives were designed to transmit information to the optimizers. I
| don't believe either in the compiler doing all the analysis and all
| the optimizations alone, because some information cannot be determined
| so easily down in the middle end, and for this purpose STAPL provides
| high level constructs that avoids the programmer to stick his fingers
| on the low level details. 

I think OpenMP directives target way too low level constructs than is
appropriate to write straight high-level programs; and OpenMP is just
one model, there are equally good out there.

| I find the ideas behind STAPL very exciting
| because it allows intuitive description of the parallelism without
| extending the syntax of the C++ language, as you said, you don't have
| to teach the compiler the semantics of the new constructs that you put
| in the library.

Yes, and use of separate transformation tools that understand the
domain specific semantics of the 

  But probably this is the whole bad point about all
| this approach, ie. the reason it is so difficult to reuse the high
| level information down in the compiler...

Not if you have a cooperation between transformation tools and the
compiler -- which is one of the main reasons The Pivot has been
launched, and yes there are similar projects around :-)

| > There is another project initiated by Bjarne Stroustrup, which I know
| > more about (certainly more than STAPL :-)), called "The Pivot".
| > 
| > The Pivot is a general framework for static analysis and
| > transformations  of C++ programs.  One of the main goal is to provide
| > direct support for distributed and parallel computations in C++.
| > We believe that support through library is much more scalable, more
| > portable, more general than support through CPP and language
| > extensions that would require most of compiler rewrite.
| > 
| > The Pivot relies on a (potentially standard) internal program
| > representation.  Here is a 30-min slides overview of the 
| > "The Pivot"/IPR structure.
| > 
| >    http://www-unix.mcs.anl.gov/workshops/DSLOpt/Talks/DosReis.pdf
| > 
| > So, you can see my bias:  Keep the compiler simple, delegate
| > domain-specific semantics to domain-specific tools;  Keep the
| > abstraction high level, and delegate byte fiddling to the computer
| > scientists and equations to physisists/numericists.
| > 
| > Keeping the compiler simple also make it easier to figure out bugs and
| > fix them.  Just take GCC as an example :-)
| 
| Up to prototyping tools you're right, but just thinking at instruction
| level parallelism for which we're far from producing good code these
| days, I'm sure that you didn't meant the last joke about reducing the
| role of the compiler in the chain.

I meant and still mean to stress on the necessity of 

  (1) high-level abstractions and separation of concern at that level;
  (2) improving the code generation based on information passed to the
      compiler -- where it does not have to mess with the higher levels.
 
And no, I do not think that OpenMP with the compiler having to do all
the work is the way to go.

| As a last remark, I would like to say that I'm not very enthusiastic
| on having OpenMP in GCC, but this is a norm that, in my opinion, goes

you know, there are many standards out there...

Allowing programmers to communicate more information for code
generation purpose is good -- I don't think anyone is saying that is a
bad idea.  However, I'm less convinced that the way to go is for GCC
to pick that particular model.

-- Gaby