Help with linker options

[Nikolaus Dunn <nikdunn1979 at gmail dot com>]

Hi guys,

I'm trying to get something tricky working, and I think I'm close but
I can't seem to get the linker options quite right. I basically have
two shared objects that each contain implementations of an allocator
type with the same name. What I want is for the code in library A to
link against the implementation of the allocator compiled into library
A and library B to link against the implementation of the allocator in
library B.

I started by using -Bsymbolic and -Bsymbolic-functions to try to get
each library to use its own internal symbols.

The first issue I ran into after that was that by dynamically linking
against libstdc++, code in std::basic_streambuf<char,
std::char_traits<char> >::xsputn(char const*, long) was dynamically
calling libstdc++ which was then calling the wrong implementation of
the allocator.

After adding the -static-libstdc++ flag at link time, it now bypasses
libstdc++ correctly, but manages to directly call the other library's
implementation.

Here is the offending stack:
.../libB.so(+0xad8f9)[0x7f8bdfc758f9]
.../libB.so(_ZNSbIcSt11char_traitsIcE18tracking_allocatorIcEE7reserveEm+0x26)[0x7f8bdfc88a16]
.../libB.so(_ZNSt15basic_stringbufIcSt11char_traitsIcE18tracking_allocatorIcEE8overflowEi+0x9e)[0x7f8bdfc88b4e]
.../libA.so(_ZNSt15basic_streambufIcSt11char_traitsIcEE6xsputnEPKcl+0x89)[0x7f8bde6a94c9]
.../libA.so(_ZSt16__ostream_insertIcSt11char_traitsIcEERSt13basic_ostreamIT_T0_ES6_PKS3_l+0x17e)[0x7f8bde69f4be]

I dumped the assembly code for the offending function and I get this:

Dump of assembler code for function
_ZNSt15basic_streambufIcSt11char_traitsIcEE6xsputnEPKcl:
   0x00000000002ab440 <+0>:    test   %rdx,%rdx
   0x00000000002ab443 <+3>:    push   %r14
   0x00000000002ab445 <+5>:    push   %r13
   0x00000000002ab447 <+7>:    push   %r12
   0x00000000002ab449 <+9>:    push   %rbp
   0x00000000002ab44a <+10>:    push   %rbx
   0x00000000002ab44b <+11>:    jle    0x2ab4e0
<_ZNSt15basic_streambufIcSt11char_traitsIcEE6xsputnEPKcl+160>
   0x00000000002ab451 <+17>:    mov    %rdi,%r12
   0x00000000002ab454 <+20>:    mov    %rsi,%r14
   0x00000000002ab457 <+23>:    mov    %rdx,%r13
   0x00000000002ab45a <+26>:    xor    %ebx,%ebx
   0x00000000002ab45c <+28>:    nopl   0x0(%rax)
   0x00000000002ab460 <+32>:    mov    0x28(%r12),%rdi
   0x00000000002ab465 <+37>:    mov    0x30(%r12),%rax
   0x00000000002ab46a <+42>:    sub    %rdi,%rax
   0x00000000002ab46d <+45>:    jne    0x2ab490
<_ZNSt15basic_streambufIcSt11char_traitsIcEE6xsputnEPKcl+80>
   0x00000000002ab46f <+47>:    mov    (%r12),%rax
   0x00000000002ab473 <+51>:    mov    0x68(%rax),%rax
   0x00000000002ab477 <+55>:    cmp    0x2928ea(%rip),%rax        # 0x53dd68
   0x00000000002ab47e <+62>:    jne    0x2ab4c0
<_ZNSt15basic_streambufIcSt11char_traitsIcEE6xsputnEPKcl+128>
   0x00000000002ab480 <+64>:    mov    %rbx,%rax
   0x00000000002ab483 <+67>:    pop    %rbx
   0x00000000002ab484 <+68>:    pop    %rbp
   0x00000000002ab485 <+69>:    pop    %r12
   0x00000000002ab487 <+71>:    pop    %r13
   0x00000000002ab489 <+73>:    pop    %r14
   0x00000000002ab48b <+75>:    retq
   0x00000000002ab48c <+76>:    nopl   0x0(%rax)
   0x00000000002ab490 <+80>:    mov    %r13,%rbp
   0x00000000002ab493 <+83>:    mov    %r14,%rsi
   0x00000000002ab496 <+86>:    sub    %rbx,%rbp
   0x00000000002ab499 <+89>:    cmp    %rax,%rbp
   0x00000000002ab49c <+92>:    cmovg  %rax,%rbp
   0x00000000002ab4a0 <+96>:    mov    %rbp,%rdx
   0x00000000002ab4a3 <+99>:    add    %rbp,%rbx
   0x00000000002ab4a6 <+102>:    callq  0x98810 <memcpy@plt>
   0x00000000002ab4ab <+107>:    add    %rbp,0x28(%r12)
   0x00000000002ab4b0 <+112>:    cmp    %rbx,%r13
   0x00000000002ab4b3 <+115>:    jle    0x2ab480
<_ZNSt15basic_streambufIcSt11char_traitsIcEE6xsputnEPKcl+64>
   0x00000000002ab4b5 <+117>:    add    %rbp,%r14
   0x00000000002ab4b8 <+120>:    jmp    0x2ab46f
<_ZNSt15basic_streambufIcSt11char_traitsIcEE6xsputnEPKcl+47>
   0x00000000002ab4ba <+122>:    nopw   0x0(%rax,%rax,1)
   0x00000000002ab4c0 <+128>:    movzbl (%r14),%esi
   0x00000000002ab4c4 <+132>:    mov    %r12,%rdi
   0x00000000002ab4c7 <+135>:    callq  *%rax
   0x00000000002ab4c9 <+137>:    cmp    $0xffffffffffffffff,%eax
   0x00000000002ab4cc <+140>:    je     0x2ab480
<_ZNSt15basic_streambufIcSt11char_traitsIcEE6xsputnEPKcl+64>
   0x00000000002ab4ce <+142>:    add    $0x1,%rbx
   0x00000000002ab4d2 <+146>:    add    $0x1,%r14
   0x00000000002ab4d6 <+150>:    cmp    %rbx,%r13
   0x00000000002ab4d9 <+153>:    jg     0x2ab460
<_ZNSt15basic_streambufIcSt11char_traitsIcEE6xsputnEPKcl+32>
   0x00000000002ab4db <+155>:    jmp    0x2ab480
<_ZNSt15basic_streambufIcSt11char_traitsIcEE6xsputnEPKcl+64>
   0x00000000002ab4dd <+157>:    nopl   (%rax)
   0x00000000002ab4e0 <+160>:    xor    %ebx,%ebx
   0x00000000002ab4e2 <+162>:    jmp    0x2ab480
<_ZNSt15basic_streambufIcSt11char_traitsIcEE6xsputnEPKcl+64>
End of assembler dump.

The only call I see here that might be related is callq *%rax, but I'm
not sure if I'm even doing this right.

Both libA and libB have the symbol:
_ZNSt15basic_stringbufIcSt11char_traitsIcE18tracking_allocatorIcEE8overflowEi

How do I get libA to use its own version of that symbol? I thought
that was what -Bsymbolic-functions was supposed to do.

Thanks,
Nik