Re: OpenACC Profiling Interface: 'acc_register_library'

[Thomas Schwinge <thomas at codesourcery dot com>]

Hi Jakub!

On Thu, 16 May 2019 17:54:23 +0200, Jakub Jelinek <jakub@redhat.com> wrote:
> On Thu, May 16, 2019 at 05:21:56PM +0200, Thomas Schwinge wrote:
> > > Jakub, would you please especially review the non-OpenACC-specific
> > > changes here, including the libgomp ABI changes?
> > 
> > Given a baseline that I've not yet posted ;-) would you please anyway
> > have a look at the following changes?  Is it OK to add/handle the
> > 'acc_register_library' symbol in this way?  The idea behind that one is
> > that you dynamically (including via 'LD_PRELOAD') link your code against
> > a "library" providing an implementation of 'acc_register_library', or
> > even define it in your user code (see the test case below), and then upon
> > initialization, "The OpenACC runtime will invoke 'acc_register_library',
> > passing [...]".
> 
> Ugh, it is a mess

;-P Hah, I was very sure that you'd say something like that!

> (but then, seems OMPT has the same mess with
> ompt_start_tool symbol).

..., but at least it's not OpenACC alone.  ;-)

> It is nasty to call acc_register_library from initialization of the OpenMP
> library, similarly to nastyness of calling ompt_start_tool from
> initialization of the OpenACC library, neither of those symbols is reserved
> to the implementation generally.
> Can't we not do anything for -fopenacc or -fopenmp and have
> -fopenacc-profile or -fopenmpt options that would link in another shared
> library which just provides that symbol and calls it from its
> initialization?

At least for OpenACC, I don't think we'll want an additional/separate
command-line flag, but yes, a separate library that only gets linked in
for explicit '-fopenacc' would've been my next idea, too.  This should be
easy to do GCC spec-wise, and also in the libgomp Automake build system.

> The dummy implementation would be __attribute__((weak))
> and would dlsym (RTLD_NEXT, "...") and call that if it returns non-NULL,
> so even if that library happens to be linked before whatever library
> implements the user symbol.
> Looking at what libomp does for ompt_start_tool, for Darwin they don't use
> a weak symbol and instead just dlsym(RTLD_DEFAULT, "...") in the
> library ctor, for Linux they have a weak definition that does dlsym
> (RTLD_NEXT, "...") and for Windows use something yet different.

Will that work for the case of static linking, though?  OpenACC for
"Statically-Linked Library Initialization" describes that "A tools
library can be compiled and linked directly into the application. If the
library provides an external routine 'acc_register_library' [...], the
runtime will invoke that routine to initialize the library".

If the proposed scheme won't work, we'll probably have to make the
runtime (libgomp) aware whether an explicit compile-time '-fopenacc' flag
had been specified, and only if yes, at run-time then invoke
'acc_register_library'?

Anyway, I'll defer the actual implementation for later.

But I'll still now include in the commit that I'm preparing the
'acc_register_library' prototype in <openacc.h>, and also its symbol
version, because these things apply no matter whether we now call that
function from 'goacc_profiling_initialize' or not.

Does the 'acc_register_library' symbol version need to be backed by a
(stub) function definition?  It builds without, but it doesn't appear in
'readelf --dyn-syms x86_64-pc-linux-gnu/libgomp/.libs/libgomp.so'; is
that OK or not?


> >     --- libgomp/libgomp.map
> >     +++ libgomp/libgomp.map
> >     @@ -469,6 +469,7 @@ OACC_2.5 {
> >      	acc_prof_lookup;
> >      	acc_prof_register;
> >      	acc_prof_unregister;
> >     +	acc_register_library;
> >      	acc_update_device_async;
> >      	acc_update_device_async_32_h_;
> >      	acc_update_device_async_64_h_;
> 
> You certainly never want to add something to a symbol version
> that has been shipped in a release compiler already.

Thanks, fixed.


Grüße
 Thomas