Created attachment 42150 [details] Minimal test case The following self-contained C++ results in an invalid free.
Pressed submit accidentally. Sorry. OS: Arch Linux Compile flags: g++ -std=c++17 -O1 -D_GLIBCXX_USE_CXX11_ABI=0 -flto ./abi_crash.cpp G++ version: $ g++ -v Using built-in specs. COLLECT_GCC=g++ COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-pc-linux-gnu/7.2.0/lto-wrapper Target: x86_64-pc-linux-gnu Configured with: /build/gcc-multilib/src/gcc/configure --prefix=/usr --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=https://bugs.archlinux.org/ --enable-languages=c,c++,ada,fortran,go,lto,objc,obj-c++ --enable-shared --enable-threads=posix --enable-libmpx --with-system-zlib --with-isl --enable-__cxa_atexit --disable-libunwind-exceptions --enable-clocale=gnu --disable-libstdcxx-pch --disable-libssp --enable-gnu-unique-object --enable-linker-build-id --enable-lto --enable-plugin --enable-install-libiberty --with-linker-hash-style=gnu --enable-gnu-indirect-function --enable-multilib --disable-werror --enable-checking=release --enable-default-pie --enable-default-ssp Thread model: posix gcc version 7.2.0 (GCC) Running the compiled code under valgrind results in: valgrind ./a.out ==23743== Memcheck, a memory error detector ==23743== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==23743== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info ==23743== Command: ./a.out ==23743== ==23743== Invalid free() / delete / delete[] / realloc() ==23743== at 0x4C2E64B: operator delete(void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==23743== by 0x4F02623: _M_dispose (basic_string.h:3155) ==23743== by 0x4F02623: ~basic_string (basic_string.h:3498) ==23743== by 0x4F02623: std::basic_stringbuf<char, std::char_traits<char>, std::allocator<char> >::overflow(int) (sstream.tcc:113) ==23743== by 0x108C1C: main (in /var/tmp/a.out) ==23743== Address 0x30a0a0 is 0 bytes inside data symbol "_ZNSs4_Rep20_S_empty_rep_storageE" ==23743== { <insert_a_suppression_name_here> Memcheck:Free fun:_ZdlPv fun:_M_dispose fun:~basic_string fun:_ZNSt15basic_stringbufIcSt11char_traitsIcESaIcEE8overflowEi fun:main } ==23743== ==23743== HEAP SUMMARY: ==23743== in use at exit: 0 bytes in 0 blocks ==23743== total heap usage: 2 allocs, 3 frees, 73,241 bytes allocated ==23743== ==23743== All heap blocks were freed -- no leaks are possible ==23743== ==23743== For counts of detected and suppressed errors, rerun with: -v ==23743== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Created attachment 42151 [details] preprocessed file that triggers the issue from -save-temps
The problem disappears in the following cases: - LTO is not enabled - libstdc++ is statically linked - If the C++11 ABI is used - Lower optimisation level - "Downgrade" to c++14 or c++11
Backtrace from GDB: Starting program: /var/tmp/a.out *** Error in `/var/tmp/a.out': free(): invalid pointer: 0x00000001002020a0 *** Program received signal SIGABRT, Aborted. 0x00007faabc0818a0 in raise () from /usr/lib/libc.so.6 (gdb) bt #0 0x00007faabc0818a0 in raise () from /usr/lib/libc.so.6 #1 0x00007faabc082f09 in abort () from /usr/lib/libc.so.6 #2 0x00007faabc0c4517 in __libc_message () from /usr/lib/libc.so.6 #3 0x00007faabc0cac84 in malloc_printerr () from /usr/lib/libc.so.6 #4 0x00007faabc0cc599 in _int_free () from /usr/lib/libc.so.6 #5 0x00007faabca30624 in std::string::_Rep::_M_dispose (__a=..., this=<optimized out>) at /build/gcc-multilib/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.h:3155 #6 std::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string (this=0x7fffffffbdb0, __in_chrg=<optimized out>) at /build/gcc-multilib/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.h:3498 #7 std::basic_stringbuf<char, std::char_traits<char>, std::allocator<char> >::overflow (this=0x7fffffffbe00, __c=97) at /build/gcc-multilib/src/gcc-build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/sstream.tcc:113 #8 0x0000000100000c1d in main ()
I encountered the same bug : https://stackoverflow.com/questions/46299811/flto-crash-with-gcc7-2 I have a crash with getline in the following code file. I built gcc7.2 because system updates are not available. Minimal example : #include <iostream> int main(int argc, char *argv[]) { std::string line; while (std::getline(std::cin, line)) { } return 0; } On the following lines, GCC_INSTALL_DIR represents the directory where my own gcc is installed Output : ./a.out a *** Error in `./a.out': free(): invalid pointer: 0x0000000000602200 *** ======= Backtrace: ========= /lib64/libc.so.6(+0x7cfe1)[0x7f392a8f3fe1] [GCC_INSTALL_DIR]/generated/lib64/libstdc++.so.6(_ZNSs7reserveEm+0x85)[0x7f392b221cd5] [GCC_INSTALL_DIR]/generated/lib64/libstdc++.so.6(_ZSt7getlineIcSt11char_traitsIcESaIcEERSt13basic_istreamIT_T0_ES7_RSbIS4_S5_T1_ES4_+0x175)[0x7f392b1fa675] ./a.out[0x40120d] ./a.out[0x4010a9] /lib64/libc.so.6(__libc_start_main+0xf5)[0x7f392a898b15] ./a.out[0x400f29] ======= Memory map: ======== 00400000-00402000 r-xp 00000000 08:05 3312183238 [redacted]/a.out 00601000-00602000 r--p 00001000 08:05 3312183238 [redacted]/a.out 00602000-00603000 rw-p 00002000 08:05 3312183238 [redacted]/a.out 008a6000-008d8000 rw-p 00000000 00:00 0 [heap] 7f3924000000-7f3924021000 rw-p 00000000 00:00 0 7f3924021000-7f3928000000 ---p 00000000 00:00 0 7f392a877000-7f392aa2d000 r-xp 00000000 08:03 201329280 /usr/lib64/libc-2.17.so 7f392aa2d000-7f392ac2d000 ---p 001b6000 08:03 201329280 /usr/lib64/libc-2.17.so 7f392ac2d000-7f392ac31000 r--p 001b6000 08:03 201329280 /usr/lib64/libc-2.17.so 7f392ac31000-7f392ac33000 rw-p 001ba000 08:03 201329280 /usr/lib64/libc-2.17.so 7f392ac33000-7f392ac38000 rw-p 00000000 00:00 0 7f392ac38000-7f392ac4e000 r-xp 00000000 08:05 93316545 [GCC_INSTALL_DIR]/generated/lib64/libgcc_s.so.1 7f392ac4e000-7f392ae4d000 ---p 00016000 08:05 93316545 [GCC_INSTALL_DIR]/generated/lib64/libgcc_s.so.1 7f392ae4d000-7f392ae4e000 r--p 00015000 08:05 93316545 [GCC_INSTALL_DIR]/generated/lib64/libgcc_s.so.1 7f392ae4e000-7f392ae4f000 rw-p 00016000 08:05 93316545 [GCC_INSTALL_DIR]/generated/lib64/libgcc_s.so.1 7f392ae4f000-7f392af50000 r-xp 00000000 08:03 201329288 /usr/lib64/libm-2.17.so 7f392af50000-7f392b14f000 ---p 00101000 08:03 201329288 /usr/lib64/libm-2.17.so 7f392b14f000-7f392b150000 r--p 00100000 08:03 201329288 /usr/lib64/libm-2.17.so 7f392b150000-7f392b151000 rw-p 00101000 08:03 201329288 /usr/lib64/libm-2.17.so 7f392b151000-7f392b2c3000 r-xp 00000000 08:05 93812046 [GCC_INSTALL_DIR]/generated/lib64/libstdc++.so.6.0.24 7f392b2c3000-7f392b4c2000 ---p 00172000 08:05 93812046 [GCC_INSTALL_DIR]/generated/lib64/libstdc++.so.6.0.24 7f392b4c2000-7f392b4cc000 r--p 00171000 08:05 93812046 [GCC_INSTALL_DIR]/generated/lib64/libstdc++.so.6.0.24 7f392b4cc000-7f392b4ce000 rw-p 0017b000 08:05 93812046 [GCC_INSTALL_DIR]/generated/lib64/libstdc++.so.6.0.24 7f392b4ce000-7f392b4d2000 rw-p 00000000 00:00 0 7f392b4d2000-7f392b4f3000 r-xp 00000000 08:03 201329241 /usr/lib64/ld-2.17.so 7f392b6d4000-7f392b6d9000 rw-p 00000000 00:00 0 7f392b6f0000-7f392b6f3000 rw-p 00000000 00:00 0 7f392b6f3000-7f392b6f4000 r--p 00021000 08:03 201329241 /usr/lib64/ld-2.17.so 7f392b6f4000-7f392b6f5000 rw-p 00022000 08:03 201329241 /usr/lib64/ld-2.17.so 7f392b6f5000-7f392b6f6000 rw-p 00000000 00:00 0 7fff7363e000-7fff7365f000 rw-p 00000000 00:00 0 [stack] 7fff7374b000-7fff7374d000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] Abandon (core dumped) Linked library: ldd ./a.out linux-vdso.so.1 => (0x00007ffcf9fe6000) libstdc++.so.6 => [GCC_INSTALL_DIR]/generated/lib64/libstdc++.so.6 (0x00007f04a565e000) libm.so.6 => /lib64/libm.so.6 (0x00007f04a5342000) libgcc_s.so.1 => [GCC_INSTALL_DIR]/generated/lib64/libgcc_s.so.1 (0x00007f04a512b000) libc.so.6 => /lib64/libc.so.6 (0x00007f04a4d6a000) /lib64/ld-linux-x86-64.so.2 (0x00007f04a59e00 libc version: ldd --version ldd (GNU libc) 2.1700) Configure option for gcc7.2 $OLD_PWD/gcc-7.2.0/configure --prefix=$OLD_PWD/generated --disable-multilib Compile command line: [GCC_INSTALL_DIR]/generated/bin/g++ -c -g -I[GCC_INSTALL_DIR]/generated/include/c++/7.2.0 -std=c++17 -flto -o main.o main.cpp -D_GLIBCXX_USE_CXX11_ABI=0 && [GCC_INSTALL_DIR]/generated/bin/g++ -flto main.o Valgrind output: ==28919== Memcheck, a memory error detector ==28919== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al. ==28919== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info ==28919== Command: ./a.out ==28919== a ==28919== Invalid free() / delete / delete[] / realloc() ==28919== at 0x4C2B131: operator delete(void*) (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==28919== by 0x4F05CD4: _M_dispose (basic_string.h:3155) ==28919== by 0x4F05CD4: std::string::reserve(unsigned long) (basic_string.tcc:961) ==28919== by 0x4EDE674: push_back (basic_string.h:4109) ==28919== by 0x4EDE674: operator+= (basic_string.h:3966) ==28919== by 0x4EDE674: std::basic_istream<char, std::char_traits<char> >& std::getline<char, std::char_traits<char>, std::allocator<char> >(std::basic_istream<char, std::char_traits<char> >&, std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, char) (istream-string.cc:168) ==28919== by 0x40120C: std::basic_istream<char, std::char_traits<char> >& std::getline<char, std::char_traits<char>, std::allocator<char> >(std::basic_istream<char, std::char_traits<char> >&, std::basic_string<char, std::char_traits<char>, std::allocator<char> >&) (in /home/elie/dev/a.out) ==28919== by 0x4010A8: main (in /home/elie/dev/a.out) ==28919== Address 0x602200 is 0 bytes inside data symbol "_ZNSs4_Rep20_S_empty_rep_storageE" ==28919== a a ==28919== ==28919== HEAP SUMMARY: ==28919== in use at exit: 72,704 bytes in 1 blocks ==28919== total heap usage: 2 allocs, 2 frees, 72,730 bytes allocated ==28919== ==28919== LEAK SUMMARY: ==28919== definitely lost: 0 bytes in 0 blocks ==28919== indirectly lost: 0 bytes in 0 blocks ==28919== possibly lost: 0 bytes in 0 blocks ==28919== still reachable: 72,704 bytes in 1 blocks ==28919== suppressed: 0 bytes in 0 blocks ==28919== Rerun with --leak-check=full to see details of leaked memory ==28919== ==28919== For counts of detected and suppressed errors, rerun with: -v ==28919== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 1 from 1) Removing `flto`, `_GLIBCXX_USE_CXX11_ABI=0` or changing `-std=c++17` for `-std=c++14` resolves the crash.
Another workaround is to disable the explicit instantiation declarations: // Include a libstdc++ header, so config macros are defined: #include <ciso646> // Disable explicit insantiation declarations in later includes: #undef _GLIBCXX_EXTERN_TEMPLATE #include <sstream> int main() { std::stringbuf myStringBuf{}; myStringBuf.sputc('a'); return 0; } This suggests the explicit instantiations in libstdc++.so are incompatible with the C++17 declarations in the headers.
Although that doesn't seem to work for the testcase in comment 5
The bug is that the symbols in lisbtdc++.so are attempting to delete the _S_empty_rep_storage object, which suggests that this check is false when it should be true: this != &_S_empty_rep() That suggests we have multiple definitions of the empty rep symbol, which aren't being combined.
(In reply to Jonathan Wakely from comment #8) > The bug is that the symbols in lisbtdc++.so are attempting to delete the > _S_empty_rep_storage object, which suggests that this check is false when it > should be true: > > this != &_S_empty_rep() > > That suggests we have multiple definitions of the empty rep symbol, which > aren't being combined. Confirmed by making the following change and rebuilding the library: --- a/libstdc++-v3/include/bits/basic_string.h +++ b/libstdc++-v3/include/bits/basic_string.h @@ -3157,6 +3157,7 @@ _GLIBCXX_END_NAMESPACE_CXX11 // _S_empty_rep_storage is never modified and the punning should // be reasonably safe in this case. void* __p = reinterpret_cast<void*>(&_S_empty_rep_storage); + __builtin_printf("%p\n", __p); return *reinterpret_cast<_Rep*>(__p); } This produces two different addresses, the first one is the address of the empty rep in the main function, the others are the address inside libstdc++.so (where the delete happens): 0x6023c0 0x7fe314376100 0x7fe314376100 0x7fe314376100 ================================================================= ==29064==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x0000006023c0 in thread T0 #0 0x7fe314457fd0 in operator delete(void*) (/lib64/libasan.so.4+0xe0fd0) #1 0x7fe3140ba291 in __gnu_cxx::new_allocator<char>::deallocate(char*, unsigned long) /home/jwakely/src/gcc/build/x86_64-pc-linux-gnu/libstdc++-v3/include/ext/new_allocator.h:125 #2 0x7fe3140b9745 in std::string::_Rep::_M_destroy(std::allocator<char> const&) /home/jwakely/src/gcc/build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.tcc:899 #3 0x7fe3140b96f7 in std::string::_Rep::_M_dispose(std::allocator<char> const&) /home/jwakely/src/gcc/build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.h:3252 #4 0x7fe3140b635f in std::string::reserve(unsigned long) /home/jwakely/src/gcc/build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.tcc:961 #5 0x7fe3140b6c9a in std::string::push_back(char) /home/jwakely/src/gcc/build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.h:4239 #6 0x7fe3140b66a6 in std::string::operator+=(char) /home/jwakely/src/gcc/build/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.h:4090 #7 0x7fe31408891f in std::basic_istream<char, std::char_traits<char> >& std::getline<char, std::char_traits<char>, std::allocator<char> >(std::basic_istream<char, std::char_traits<char> >&, std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, char) /home/jwakely/src/gcc/gcc/libstdc++-v3/src/c++98/istream-string.cc:168 #8 0x40106a in main (/tmp/a.out+0x40106a) #9 0x7fe3136ec509 in __libc_start_main (/lib64/libc.so.6+0x20509) #10 0x401389 in _start (/tmp/a.out+0x401389) 0x0000006023c0 is located 0 bytes inside of global variable '_S_empty_rep_storage' defined in '/home/jwakely/gcc/8/include/c++/8.0.0/bits/basic_string.tcc:506:5' (0x6023c0) of size 32 SUMMARY: AddressSanitizer: bad-free (/lib64/libasan.so.4+0xe0fd0) in operator delete(void*) ==29064==ABORTING
Reproducible for any version since 4.6.0, without C++17 or optimization, just LTO: #define _GLIBCXX_USE_CXX11_ABI 0 #include <bits/c++config.h> #undef _GLIBCXX_EXTERN_TEMPLATE #include <iostream> int main() { std::string s; std::cin >> s; } $ g++ os.cc -flto $ echo d | ./a.out *** Error in `./a.out': free(): invalid pointer: 0x00000000006021a0 *** ======= Backtrace: ========= /lib64/libc.so.6(+0x7c8dc)[0x7fe1d6f188dc] /lib64/libc.so.6(+0x87789)[0x7fe1d6f23789] /lib64/libc.so.6(cfree+0x16e)[0x7fe1d6f290ee] /lib64/libstdc++.so.6(_ZNSs7reserveEm+0xa5)[0x7fe1d786e465] /lib64/libstdc++.so.6(_ZStrsIcSt11char_traitsIcESaIcEERSt13basic_istreamIT_T0_ES7_RSbIS4_S5_T1_E+0x217)[0x7fe1d7844787] ./a.out[0x4008c2] /lib64/libc.so.6(__libc_start_main+0xea)[0x7fe1d6ebc50a] ./a.out[0x40074a] ======= Memory map: ======== 00400000-00402000 r-xp 00000000 00:29 43484686 /tmp/a.out 00601000-00602000 r--p 00001000 00:29 43484686 /tmp/a.out 00602000-00603000 rw-p 00002000 00:29 43484686 /tmp/a.out 0084d000-0087f000 rw-p 00000000 00:00 0 [heap] 7fe1d0000000-7fe1d0021000 rw-p 00000000 00:00 0 7fe1d0021000-7fe1d4000000 ---p 00000000 00:00 0 7fe1d6e9c000-7fe1d7063000 r-xp 00000000 fd:00 1971889 /usr/lib64/libc-2.25.so 7fe1d7063000-7fe1d7263000 ---p 001c7000 fd:00 1971889 /usr/lib64/libc-2.25.so 7fe1d7263000-7fe1d7267000 r--p 001c7000 fd:00 1971889 /usr/lib64/libc-2.25.so 7fe1d7267000-7fe1d7269000 rw-p 001cb000 fd:00 1971889 /usr/lib64/libc-2.25.so 7fe1d7269000-7fe1d726d000 rw-p 00000000 00:00 0 7fe1d726d000-7fe1d7283000 r-xp 00000000 fd:00 1977375 /usr/lib64/libgcc_s-7-20170915.so.1 7fe1d7283000-7fe1d7482000 ---p 00016000 fd:00 1977375 /usr/lib64/libgcc_s-7-20170915.so.1 7fe1d7482000-7fe1d7483000 r--p 00015000 fd:00 1977375 /usr/lib64/libgcc_s-7-20170915.so.1 7fe1d7483000-7fe1d7484000 rw-p 00016000 fd:00 1977375 /usr/lib64/libgcc_s-7-20170915.so.1 7fe1d7484000-7fe1d7599000 r-xp 00000000 fd:00 1971970 /usr/lib64/libm-2.25.so 7fe1d7599000-7fe1d7798000 ---p 00115000 fd:00 1971970 /usr/lib64/libm-2.25.so 7fe1d7798000-7fe1d7799000 r--p 00114000 fd:00 1971970 /usr/lib64/libm-2.25.so 7fe1d7799000-7fe1d779a000 rw-p 00115000 fd:00 1971970 /usr/lib64/libm-2.25.so 7fe1d779a000-7fe1d7914000 r-xp 00000000 fd:00 1977857 /usr/lib64/libstdc++.so.6.0.24 7fe1d7914000-7fe1d7b14000 ---p 0017a000 fd:00 1977857 /usr/lib64/libstdc++.so.6.0.24 7fe1d7b14000-7fe1d7b1e000 r--p 0017a000 fd:00 1977857 /usr/lib64/libstdc++.so.6.0.24 7fe1d7b1e000-7fe1d7b20000 rw-p 00184000 fd:00 1977857 /usr/lib64/libstdc++.so.6.0.24 7fe1d7b20000-7fe1d7b23000 rw-p 00000000 00:00 0 7fe1d7b23000-7fe1d7b4a000 r-xp 00000000 fd:00 1971811 /usr/lib64/ld-2.25.so 7fe1d7d0c000-7fe1d7d10000 rw-p 00000000 00:00 0 7fe1d7d46000-7fe1d7d49000 rw-p 00000000 00:00 0 7fe1d7d49000-7fe1d7d4a000 r--p 00026000 fd:00 1971811 /usr/lib64/ld-2.25.so 7fe1d7d4a000-7fe1d7d4c000 rw-p 00027000 fd:00 1971811 /usr/lib64/ld-2.25.so 7ffc25685000-7ffc256a7000 rw-p 00000000 00:00 0 [stack] 7ffc2576b000-7ffc2576d000 r--p 00000000 00:00 0 [vvar] 7ffc2576d000-7ffc2576f000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] Aborted (core dumped)
Confirmed, LTO really does a local object from the storage: $ g++ -flto ice.cc && readelf -s a.out --wide | grep rep_storage 77: 00000000006021a0 32 OBJECT LOCAL DEFAULT 26 _ZNSs4_Rep20_S_empty_rep_storageE $ g++ ice.cc && readelf -s a.out --wide | grep rep_storage 25: 0000000000602180 32 OBJECT UNIQUE DEFAULT 26 _ZNSs4_Rep20_S_empty_rep_storageE 116: 0000000000602180 32 OBJECT UNIQUE DEFAULT 26 _ZNSs4_Rep20_S_empty_rep_storageE I'll take a look.
N.B. this bug has been latent for years. The reason people are only seeing it now is that usually the "extern template class std::basic_string<char>" explicit instantiation declarations prevent this symbol being emitted in user's object files, so the definition in the library is used. When compiling with -std=c++17 that extern template is not declared, so the compiler emits a definition of that symbol in every object that uses the old std::string.
Looking at resolution file of the compilation unit: $ cat -- -lm.res 1 ice.o 39 ... 1022 a2617b1cb04da8c1 PREVAILING_DEF_IRONLY _ZNSs4_Rep20_S_empty_rep_storageE ... I'm not fully familiar with symbol resolution, but this looks fishy. $ readelf -s /usr/lib64/gcc/x86_64-suse-linux/7/libstdc++.so --wide | grep _ZNSs4_Rep20_S_empty_rep_storageE 4310: 0000000000388d40 32 OBJECT UNIQUE DEFAULT 28 _ZNSs4_Rep20_S_empty_rep_storageE@@GLIBCXX_3.4 Any ideas Jonathan?
I have no idea what PREVAILING_DEF_IRONLY means, but a web search found https://sourceware.org/bugzilla/show_bug.cgi?id=12370 -- I can't tell if that's relevant.
(In reply to Jonathan Wakely from comment #12) > N.B. this bug has been latent for years. The reason people are only seeing > it now is that usually the "extern template class std::basic_string<char>" > explicit instantiation declarations prevent this symbol being emitted in > user's object files, so the definition in the library is used. > > When compiling with -std=c++17 that extern template is not declared, so the > compiler emits a definition of that symbol in every object that uses the old > std::string. LDPR_PREVAILING_DEF_IRONLY means that the symbol in our translation unit is prevailing definition with no reference. However it's not true as libstdc++ shared library is having it's own symbol. That's what causes the segfault. Shouldn't one have different versions of libstdc++.so in order to either have a single global symbol or definitions in user's object files?
(In reply to Martin Liška from comment #15) > Shouldn't one have different versions of libstdc++.so in order to either > have a single global symbol or definitions in user's object files? No, it's normal to have multiple definitions of C++ template instantiations, the linker should resolve them and only keep one.
So another observation, it's not related to STB_GNU_UNIQUE. Using --disable-gnu-unique-object: $ readelf -s /home/marxin/bin/gcc3/lib64/libstdc++.so.6 --wide | grep storage 1431: 000000000038dd60 32 OBJECT WEAK DEFAULT 27 _ZNSs4_Rep20_S_empty_rep_storageE@@GLIBCXX_3.4 4467: 000000000038dd80 32 OBJECT WEAK DEFAULT 27 _ZNSbIwSt11char_traitsIwESaIwEE4_Rep20_S_empty_rep_storageE@@GLIBCXX_3.4 3633: 000000000038dd60 32 OBJECT WEAK DEFAULT 27 _ZNSs4_Rep20_S_empty_rep_storageE 4224: 000000000038dd80 32 OBJECT WEAK DEFAULT 27 _ZNSbIwSt11char_traitsIwESaIwEE4_Rep20_S_empty_rep_storageE $ grep storage pr82172-2.res 1344 a6363178d2c55d5a PREVAILING_DEF_IRONLY _ZNSs4_Rep20_S_empty_rep_storageE Still, similar problem, I will debug ld a bit tomorrow.
Issue solved, ld.bfd is responsible. Gold properly marks the symbols as: 1322 aef281150a4b024f PREVAILING_DEF_IRONLY_EXP _ZNSs4_Rep20_S_empty_rep_storageE I'm going to create binutils issue for that.
(In reply to Martin Liška from comment #18) > Issue solved, ld.bfd is responsible. Unfortunately, the same test program also crashes when built and linked on OSX. I tested with Sierra (OSX 10.12.5), gcc 7.2.0, compiling the original sample here with: g++-7 -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17 -O1 test.cpp Result: > ./a.out a.out(608,0x7fffbda8a3c0) malloc: *** error for object 0x10c68d080: pointer being freed was not allocated *** set a breakpoint in malloc_error_break to debug Abort trap: 6 I think this means the Darwin linker has a similar problem.
(In reply to Gubbins from comment #19) > (In reply to Martin Liška from comment #18) > > Issue solved, ld.bfd is responsible. > > Unfortunately, the same test program also crashes when built and linked on > OSX. > > I tested with Sierra (OSX 10.12.5), gcc 7.2.0, compiling the original sample > here with: > > g++-7 -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++17 -O1 test.cpp > > Result: > > > ./a.out > a.out(608,0x7fffbda8a3c0) malloc: *** error for object 0x10c68d080: pointer > being freed was not allocated > *** set a breakpoint in malloc_error_break to debug > Abort trap: 6 > > > I think this means the Darwin linker has a similar problem. Your failure happens even w/o LTO, am I right? But yes, the problem looks very similar to what happens for ld.bfd.
(In reply to Martin Liška from comment #20) > Your failure happens even w/o LTO, am I right? > But yes, the problem looks very similar to what happens for ld.bfd. You are right. Does anyone know how I would raise this with someone who can fix it on the Darwin side? Or could it be worked around by gcc?
(In reply to Gubbins from comment #21) > (In reply to Martin Liška from comment #20) > > Your failure happens even w/o LTO, am I right? > > But yes, the problem looks very similar to what happens for ld.bfd. > > You are right. > > Does anyone know how I would raise this with someone who can fix it on the > Darwin side? Or could it be worked around by gcc? Linker should provide precise information to GCC. The bug is fixed in binutils, I'm closing this.
I think MOVED is more appropriate, as it's handled at https://sourceware.org/bugzilla/show_bug.cgi?id=22220
> > Does anyone know how I would raise this with someone who can fix it on the > > Darwin side? Or could it be worked around by gcc? > > Linker should provide precise information to GCC. FWIW, I have raised a bug report on the Apple Developer Bug Reporter regarding the similar behaviour of the OSX linker.
(In reply to Gubbins from comment #24) > > > Does anyone know how I would raise this with someone who can fix it on the > > > Darwin side? Or could it be worked around by gcc? > > > > Linker should provide precise information to GCC. > > FWIW, I have raised a bug report on the Apple Developer Bug Reporter > regarding the similar behaviour of the OSX linker. Dave, the fix for PR 86138 might also fix this case for Darwin - could you check that please? The fix is committed to the tip of trunk and gcc-8-branch and gcc-7-branch.
If anyone is interested, I received the following response on my bug report with Apple. > This issue behaves as intended based on the following: > > The program produced by ld64 seems fine: > > [/tmp/35663253]> nm -nm foo > (undefined) external __Unwind_Resume (from libSystem) > (undefined) external __ZNSt15basic_stringbufIcSt11char_traitsIcESaIcEE8overflowEi (from libstdc++) > (undefined) external __ZNSt6localeC1Ev (from libstdc++) > (undefined) external __ZNSt6localeD1Ev (from libstdc++) > (undefined) external __ZTVSt15basic_streambufIcSt11char_traitsIcEE (from libstdc++) > (undefined) external __ZTVSt15basic_stringbufIcSt11char_traitsIcESaIcEE (from libstdc++) > (undefined) weak external __ZdlPv (from libstdc++) > (undefined) external ___gxx_personality_v0 (from libstdc++) > (undefined) external dyld_stub_binder (from libSystem) > 0000000000001000 (absolute) [referenced dynamically] external __mh_execute_header > 0000000000001dee (__TEXT,__text) weak external __ZNSt15basic_stringbufIcSt11char_traitsIcESaIcEED1Ev > 0000000000001e3f (__TEXT,__text_startup) external _main > 0000000000002060 (__DATA,__gcc_except_tab) non-external GCC_except_table0 > 0000000000002080 (__DATA,__data) weak external __ZNSs4_Rep20_S_empty_rep_storageE > > [/tmp/35663253]> dyldinfo -weak_bind foo > weak binding information: > segment section address type addend symbol > __DATA __got 0x00002010 pointer 0 __ZNSs4_Rep20_S_empty_rep_storageE > __DATA __la_symbol_ptr 0x00002040 pointer 0 __ZNSt15basic_stringbufIcSt11char_traitsIcESaIcEED1Ev > __DATA __la_symbol_ptr 0x00002058 pointer 0 __ZdlPv > __DATA __la_symbol_ptr 0x00002058 pointer 0 __ZdlPv > > The problem has to do with __ZNSs4_Rep20_S_empty_rep_storageE. That symbol is also in libstdc++.6.dylib and is expected to be coalesced. > > [/tmp/35663253]> nm -m libstdc++.6.dylib | grep __ZNSs4_Rep20_S_empty_rep_storageE > 0000000000137020 (__DATA,__pu_bss5) extern > al __ZNSs4_Rep20_S_empty_rep_storageE > > The problem is that it is not “weak” in libstdc++.6.dylib. It is a regular exported symbol. If it were weak, then at runtime dyld would coalesce it with the one in the program “foo”. > > macOS does not use “flat namespace”. It uses two level namespace where every symbol found in a dylib at build time has the dylib in which it was found recorded and at runtime dyld only looks there. The exception to this is weak symbols, where dyld looks across all dylibs and picks one, then adjusts all uses in all dylibs to use that choosen one. > > The static linker (ld64) knows those rules and when building a dylib that exports a non-weak symbol, the linker optimizes all uses within that dylib to directly use that symbol. That is what is happening in libstdc++.6.dylib. __ZNSs4_Rep20_S_empty_rep_storageE is not weak, so when libstdc++.6.dylib all uses of __ZNSs4_Rep20_S_empty_rep_storageE are directly bound to use the copy in libstdc++.6.dylib. There is nothing dyld can do at runtime to change that. > > The fix here is that __ZNSs4_Rep20_S_empty_rep_storageE needs to be weak when libstdc++.6.dylib is built.
> Dave, the fix for PR 86138 might also fix this case for Darwin - could you > check that please? I can confirm that using my homebrew-installed gcc 8.2.0 package on OSX, the issue no longer occurs. I don't know if the change made for PR 86138 is responsible but I believe 8.2.0 was the first release to include it.
Thanks, Dave. That's very helpful. (In reply to Gubbins from comment #26) > > The fix here is that __ZNSs4_Rep20_S_empty_rep_storageE needs to be weak when libstdc++.6.dylib is built. Or make sure the explicit instantiation is declared in <string> so that the symbol is not emitted in the program, and the one in libstdc++.6.dylib is always used. Which is how this bug was fixed. I need to investigate whether we have a problem with any other explicit instantiations which are not declared in headers, e.g. these ones https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=libstdc%2B%2B-v3/src/c%2B%2B11/ext11-inst.cc;h=5d16ea6ea185be25be48e030de74e3448eddf2e6;hb=HEAD
I've opened Bug 87380