Re: static linking with openmp (was: gomp - cost of threadprivate data access)

[Matevz Tadel <matevz dot tadel at cern dot ch>]

VandeVondele Joost wrote:
> If you want to statically link with gcc and openmp (i.e. combine 
> -fopenmp -static), you should have a look at:
>
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=39176#c7
>
> basically you have to link pthreads yourself as
>
> -Wl,--whole-archive -lpthread -Wl,--no-whole-archive

Thanks for the hint.

Nevertheless, I seem to have a mess with glibc - I get a crash soon after a 
non-main thread starts writing out some data. Here is an example stack trace:

#0  0x0000000001a69974 in vsnprintf ()
#1  0x0000000001a61361 in snprintf ()
#2  0x000000000199c20d in write_float (dtp=0x7ffff6816540, f=0x87dcbd8,
    source=<value optimized out>, len=<value optimized out>)
    at /opt/gcc-4.3-svn/libgfortran/io/write_float.def:792
#3  0x0000000001996f79 in formatted_transfer_scalar (dtp=0x7ffff6816540,
    type=BT_REAL, p=0x7ffff681670c, len=4, size=4)
    at /opt/gcc-4.3-svn/libgfortran/io/transfer.c:1116
#4  0x0000000001997675 in formatted_transfer (dtp=0x7ffff6816540,
    type=BT_REAL, p=<value optimized out>, kind=4, size=4, nelems=1)
    at /opt/gcc-4.3-svn/libgfortran/io/transfer.c:1378
#5  0x0000000000484247 in g3init_ ()
#6  0x00000000004f4d89 in g3cinit_ ()
#7  0x000000000042e915 in TGeant3::TGeant3 ()
#8  0x000000000043432c in TGeant3TGeo::TGeant3TGeo ()


I think I tried just about any possible combination of linker incantations.

For -static:
------------
g++ smain.o -o main -static \
  ../../geant4_vmc/lib/tgt_linuxx8664gcc/libexample03.a \
  ../lib/tgt_linuxx8664gcc/libgeant321.a \
  ${ROOTSYS}/lib/libRoot.a ${EXTRAROOTLIB} \
  ${ROOTSYS}/lib/libpcre.a ${ROOTSYS}/lib/libfreetype.a \
  -lgfortran -lgomp \
  -Wl,--whole-archive  -lpthread -Wl,--no-whole-archive \
  -ldl

I have to add -ldl as ROOT's core library use dlopen for plugin loading (this is 
not used in my program).

-lgomp is not really needed - which worries me.

For -dynamic:
-------------
g++ smain.o -o $@ -Wl,-Bstatic \
  ../../geant4_vmc/lib/tgt_linuxx8664gcc/libexample03.a \
  ../lib/tgt_linuxx8664gcc/libgeant321.a \
  ${ROOTSYS}/lib/libRoot.a ${EXTRAROOTLIB} \
  ${ROOTSYS}/lib/libpcre.a ${ROOTSYS}/lib/libfreetype.a \
  -lgfortran -lgomp \
  -Wl,--whole-archive  -lpthread -Wl,--no-whole-archive \
  /usr/lib64/libc.a \
  -Wl,-Bdynamic -ldl

Here I have to manually add /usr/lib64/libc.a otherwise I get undefined symbols 
(__libc_setup_tls, __syscall_error, _dl_init_static_tls and couple more _dl_xyzzs).


I also tried with -fopenmp and -pthread in various places, but anytime the link 
is successful the crashes in libc are there as well.

Is there a way to switch on thread-safety in libc at the runtime?

> as the default way of linking things causes segfaults. My feeling was 
> that gcc should take care of this automagically, but this doesn't seem 
> to be the consensus.

Where was it crashing in your case?

Can you please send me the full link command-line that worked for you in the end.

Sorry for this long post ... but I don't really know what are the relevant data.

Best,
Matevz