Re: TOC errors on AIX

[Perry Smith <pedzsan at gmail dot com>]

On Jul 31, 2012, at 4:30 PM, David Edelsohn wrote:

> On Sun, Jul 29, 2012 at 12:48 PM, Perry Smith <pedzsan@gmail.com> wrote:
>> Hi,
>> 
>> This is an age old topic but I can't find how to solve it.  I've searched the past few days.
>> 
>> I'm trying to build passenger on AIX 6.1 TL07 SP03 using gcc 4.5.2 that I built myself.  I've used it for a number of months and have built many things.
>> 
>> The short question is how do I get rid of these warnings?
>> 
>>> ld: 0711-768 WARNING: Object ext/apache2/module_libboost_oxt.a[system_calls.o], section 1, function .accept:
>>>        The branch at address 0x2ac0 is not followed by a recognized no-op
>>>        or TOC-reload instruction. The unrecognized instruction is 0x7C601B78.
>> 
>> The link has many errors like this.  For whatever reason, passenger builds the same file two different ways and puts them in two different places.  Upon closer examination the file that the link is complaining about does in fact have bad code.
>> 
>> The bad compile line is:
>> 
>>> g++ -Iext -fPIC -fvisibility=hidden -DVISIBILITY_ATTRIBUTE_SUPPORTED -U__STR__ -D_THREAD_SAFE -D_LARGEFILE64_SOURCE -I/gsa/ausgsa/projects/r/ruby/apache2/include/apr-1 -I/gsa/ausgsa/projects/r/ruby/apache2/include/apr-1 -I/gsa/ausgsa/projects/r/ruby/apache2/include -D_REENTRANT -I/usr/local/include -DHASH_NAMESPACE="__gnu_cxx" -DHASH_NAMESPACE="__gnu_cxx" -DHASH_FUN_H="<hash_fun.h>" -DOXT_DISABLE_BACKTRACES -DHAS_ALLOCA_H -Wall -Wextra -Wno-unused-parameter -Wno-parentheses -Wpointer-arith -Wwrite-strings -Wno-long-long -Wno-missing-field-initializers -g -DPASSENGER_DEBUG -DBOOST_DISABLE_ASSERTS -o ext/apache2/module_libboost_oxt/oxt/system_calls.o -c ext/oxt/system_calls.cpp
>> 
>> and produces:
>> 
>>> ...
>>>       lwz 5,128(31)
>>>        bl .accept
>>>        mr 0,3
>>> ...
>> 
>> There needs to be a nop after the bl so the linker / loader can stuff in an instruction to restore the toc.
>> 
>> There are many warnings for this compile as well about the visibility.  e.g.
>> 
>>> ext/oxt/system_calls.cpp: In function 'int oxt::syscalls::accept(int, sockaddr*, socklen_t*)':
>>> ext/oxt/system_calls.cpp:209:1: warning: visibility attribute not supported in this configuration; ignored
>> 
>> 
>> The compile line that is good is:
>> 
>>> g++ -Iext  -D_REENTRANT -I/usr/local/include -DHASH_NAMESPACE="__gnu_cxx" -DHASH_NAMESPACE="__gnu_cxx" -DHASH_FUN_H="<hash_fun.h>" -DOXT_DISABLE_BACKTRACES -DHAS_ALLOCA_H -Wall -Wextra -Wno-unused-parameter -Wno-parentheses -Wpointer-arith -Wwrite-strings -Wno-long-long -Wno-missing-field-initializers -g -DPASSENGER_DEBUG -DBOOST_DISABLE_ASSERTS -o ext/common/libboost_oxt/oxt/system_calls.o -c ext/oxt/system_calls.cpp
>> 
>> which produces good code:
>> 
>>> ...
>>>        lwz 5,128(31)
>>>        bl .accept
>>>        nop
>>>        mr 0,3
>>> ...
>> 
>> The good compile has no warnings.
>> 
>> I figure it is the -fvisibility that is getting me into trouble but I am not sure if it is safe to just get rid of it.  There are also macros in the boost *.hpp files that muck with the visibility setting.
>> 
>> The link line is (if needed):
>> 
>>> g++ -shared ext/apache2/Configuration.o ext/apache2/Bucket.o ext/apache2/Hooks.o ext/apache2/mod_passenger.o -fPIC -o ext/apache2/mod_passenger.so -fPIC -fvisibility=hidden -DVISIBILITY_ATTRIBUTE_SUPPORTED -U__STR__ -D_THREAD_SAFE -D_LARGEFILE64_SOURCE -I/gsa/ausgsa/projects/r/ruby/apache2/include/apr-1 -I/gsa/ausgsa/projects/r/ruby/apache2/include/apr-1 -I/gsa/ausgsa/projects/r/ruby/apache2/include -D_REENTRANT -I/usr/local/include -DHASH_NAMESPACE="__gnu_cxx" -DHASH_NAMESPACE="__gnu_cxx" -DHASH_FUN_H="<hash_fun.h>" -DOXT_DISABLE_BACKTRACES -DHAS_ALLOCA_H -Wall -Wextra -Wno-unused-parameter -Wno-parentheses -Wpointer-arith -Wwrite-strings -Wno-long-long -Wno-missing-field-initializers -g -DPASSENGER_DEBUG -DBOOST_DISABLE_ASSERTS ext/apache2/module_libpassenger_common.a ext/apache2/module_libboost_oxt.a -fPIC -L/gsa/ausgsa/projects/r/ruby/apache2/lib -lapr-1 -Wl,-G -Wl,-brtl -L/gsa/ausgsa/projects/r/ruby/apache2/lib -laprutil-1 -lpthread
> 
> The problem most likely is -fvisibility.  AIX does not support
> visibility the way that GNU/Linux does, although GNU/Linux is evolving
> toward the symbol visibility model that AIX had all along.
> 
> Two questions are:
> 
> 1) Why does the package think that AIX supports visibility and using
> those options?

I tracked this down.  They ask "does the compiler support it?" (which gcc does)
and so they assume the platform does.  I changed the test to just return false on AIX.  

That got rid of the -fvisibility flag, 99% of my compile warnings as well as the link
warnings.

> 
> 2) Why is the package linking with -Wl,-G -Wl,-brtl?

I've never 100% understood all this and so the package probably doesn't either.
But -G implies -brtl so having both should be ok?  Or did I misread the man page?

> Basically, the package is trying to mix-and-match AIX linking behavior
> and SVR4 behavior.  It is trying to invoke the AIX linker with SVR4
> behavior, but then overriding the default SVR4 semantics and not using
> matching flags for AIX.  AIX defaults to not exporting most symbols,
> but has options to export more symbols -- matching SVR4 behavior.
> 
> The visibility options are trying to override the default SVR4 symbol
> export behavior, which asserts to the compiler that the symbols will
> be local and not need nops required by external calls.  But then the
> AIX linker is invoked with options to export more symbols and it is
> finding external definitions of the symbols.
> 
> I do not know what you mean by building two files in two different
> ways and placing them in two different locations.  Does the link line
> link against both copies?  

No, it uses only one for the link I pasted.  ext/apache2/module_libboost_oxt.a
is a normal archive of ext/apache2/*.o  I assume it uses the files in
ext/common/libboost_oxt/oxt/*.o in a different link.

> The Makefile also may be confused about
> PIC, which is the default on AIX.  The external symbol may be
> overriding a local symbol because of AIX linker semantics.

Yea.  That seems to be benign so I haven't worried about it.

> The basic problem is GNU/Linux symbol visibility behavior is becoming
> more complicated (and actually evolving toward original AIX behavior),
> but it also is using additional syntax and options in Makefiles
> without invoking AIX with compatible, matching options.

I'm hoping this is a good thing.  GNU/Linux, to me, seemed rather sloopy
and they could "link" and produce code that could eventually fail due to
a missing symbol.  The general open software developer perspective
seemed to have a disdain for AIX's wanting to know where every symbol
was going to come from at link time.

> Removing -fvisibility probably will not cause any additional problems
> because the AIX linker already is ignoring the options and intended
> behavior, but many more symbols are leaking out than the package
> intended.

Thank you for your help.  I am mostly completed the changes but are
fighting a core dump when exit is called.

Thank you again,
Perry