With the upgrade to fedora 17, I upgraded to g++ 4.7, which builds a segfault in my current project. I did a lot of research before submitting this bug, while discussing the problem with Niall Douglas on the c++sig mailing list of python. So here is what I found. First the problem itself. In inheritance.cpp:405 (the .ii file of the compilation is attached), compiled with: g++ -ftemplate-depth-128 -O0 -finline-functions -Wno-inline -Wall -std=c++11 -pthread -DBOOST_ALL_NO_LIB=1 -DBOOST_THREAD_USE_LIB=1 -DNDEBUG -I. -I/usr/include/python2.7 -c libs/python/src/object/inheritance.cpp -save-temps the value of p gets somehow lost. This is what gdb has to say to this: #0 0x0000000000405af7 in boost::python::objects::polymorphic_id_generator<Foo>::execute (p_=0x40a268) at /usr/include/boost/python/object/inheritance.hpp:43 #1 0x00007ffff7dcb684 in boost::(anonymous namespace)::convert_type (p= 0x666db0, src_t=..., dst_t=..., polymorphic=true) at /usr/src/debug/boost_1_48_0/libs/python/src/object/inheritance.cpp:405 Inspecting the code we find that p_ in execute should actually be the same as p, as it is passed literally in convert_type. However, its value magically changes. Here are the characteristics of the problem. It reoccurs with inlining force-disabled. It reoccurs with optimization and without (tried for both, the testing program and for compiling libboost_python). It does _not_ reoccur without -std=c++11 in the testing program. I am not using precompiled headers. I tried boost as delivered with fedora 17, a home-compiled version with -std=c++11 and a home-compiled version without c++11. The c++11 flag on the _library_ does not seem to influence the problem, although the value change is inside the library. However, when turning off c++11 for the testing program (going to attach the .ii too), it works fine. This behaviour did not occur with g++ 4.6.3 and -std=c++0x. Please do not hesitate to ask back if you need any more information, I'm glad if I can help. I also hope that I read the guidelines for bug reporting correctly. If you nevertheless want my testing program source attached, I will also attach that. The crashtest.ii was compiled with: g++ -Wall -g -std=c++0x -I/usr/include/python2.7 -fno-inline -save-temps /home/tester/libboost_python.so.1.48.0 -lpython2.7 crashtest.cpp -o crashtest g++ -v: Using built-in specs. COLLECT_GCC=g++ COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/4.7.0/lto-wrapper Target: x86_64-redhat-linux Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --disable-build-with-cxx --disable-build-poststage1-with-cxx --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-linker-hash-style=gnu --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto --enable-plugin --enable-initfini-array --enable-java-awt=gtk --disable-dssi --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre --enable-libgcj-multifile --enable-java-maintainer-mode --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libjava-multilib --with-ppl --with-cloog --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux Thread model: posix gcc version 4.7.0 20120507 (Red Hat 4.7.0-5) (GCC)
Created attachment 27477 [details] -save-temps output for inheritance.cpp from boost::python 1.48 Sorry for the .gz, but bugzilla refused the > 2 MB file.
Created attachment 27478 [details] -save-temps output for crashtest.cpp (my small test program) again sorry for the .gz, same as above
CC-ing
CC-ing too.
Link to the c++-sig discussion thread: http://mail.python.org/pipermail/cplusplus-sig/2012-May/016581.html For the purposes of assigning tags, this bug is C++11 mode only, GCC 4.7.x only, affects code doing virtual inheritance.
I believe I hit the same bug, using extract on a polymorphic type. In the old code (which segfaults), I used make_constructor, passing object to it, and using extract. In the new code (which does not segfault), I don't use make_constructor, just use bp::init. My segfault is gone. Maybe just luck? valgrind python test_constellation.py ==6162== Memcheck, a memory error detector ==6162== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al. ==6162== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info ==6162== Command: python test_constellation.py ==6162== ==6162== Invalid read of size 8 ==6162== at 0x11B79748: boost::python::objects::polymorphic_id_generator<QAMconstellation<double> >::execute(void*) (inheritance.hpp:43) ==6162== by 0x13F585A3: boost::(anonymous namespace)::convert_type(void*, boost::python::type_info, boost::python::type_info, bool) (in /usr/local/src/boost_1_49_0.hg.nondarray/stage/lib/libboost_python.so.1.49.0) ==6162== by 0x11B857D9: boost::python::objects::pointer_holder<std::auto_ptr<QAMconstellation<double> >, QAMconstellation<double> >::holds(boost::python::type_info, bool) (pointer_holder.hpp:150) ==6162== by 0x13F50006: boost::python::objects::find_instance_impl(_object*, boost::python::type_info, bool) (in /usr/local/src/boost_1_49_0.hg.nondarray/stage/lib/libboost_python.so.1.49.0) ==6162== by 0x13F4B304: boost::python::converter::rvalue_from_python_stage1(_object*, boost::python::converter::registration const&) (in /usr/local/src/boost_1_49_0.hg.nondarray/stage/lib/libboost_python.so.1.49.0) ==6162== by 0x11B82F02: boost::python::converter::arg_rvalue_from_python<constellation<double> const&>::arg_rvalue_from_python(_object*) (arg_from_python.hpp:299) ==6162== by 0x11B81BEA: boost::python::arg_from_python<constellation<double> const&>::arg_from_python(_object*) (arg_from_python.hpp:70) ... This particular code uses inheritance (I rarely use it). It looks something like: BOOST_PYTHON_MODULE(constellation) { boost::numpy::initialize(); class_<constellation<double>, boost::noncopyable > ("constellation_base", no_init) .def ("__call__", &apply_constellation<double>) .def ("hard_decision", &apply_hard_decision<double>) .add_property ("size", &constellation<double>::size) .add_property ("gain", &constellation<double>::gain) ; class_<QAMconstellation<double> , bases<constellation<double> > > ("QAMconstellation", no_init) .def ("__init__", make_constructor (qam_from_object<double>)) .add_property ("map", &get_map) .def_pickle(const_pickle_suite<double>()) ; ...
Interestingly, I am using no_init too, but without supplying an alternative constructor. I am not at the testing machine right now, but I thought I'd share that bit of information. Testing whether the bug reoccurs when using bp::init will follow soon!
I was able to use the VM sooner than expected, so sorry for the doublepost. I found that whether using no_init or init<>() does not make a difference in my case. To use init<>() on the base class, I had to change the pure virtual function to be non-pure, but that did not affect the result.
maybe related: https://svn.boost.org/trac/boost/ticket/6919 Had similar crash issue. Though in my case (which may well be different from the OP) rebuilding boost with new flags fixed it. If OP posts crashtest.cpp source, I'll be happy to run it through. Here's how I rebuilt the f17 rpms: # cat ~/.rpmrc optflags: x86_64 -O2 -g -std=c++11 -fno-strict-aliasing # rpmbuild --rebuild boost-1.48.0-11.fc17.src.rpm
found the OP crashtest source at the tail of .ii attachment file, that compiles and runs fine with my new rpm.
(In reply to comment #9) > maybe related: https://svn.boost.org/trac/boost/ticket/6919 > Had similar crash issue. Though in my case (which may well be different from > the OP) rebuilding boost with new flags fixed it. > If OP posts crashtest.cpp source, I'll be happy to run it through. > Here's how I rebuilt the f17 rpms: > > # cat ~/.rpmrc > optflags: x86_64 -O2 -g -std=c++11 -fno-strict-aliasing > # rpmbuild --rebuild boost-1.48.0-11.fc17.src.rpm With respect to the Boost bugtracker, if this is an aliasing bug then it's a bug in Boost. Strict aliasing isn't an optional part of the ISO standard. They shouldn't have closed that issue because -fno-strict-aliasing "fixes" the bug on 4.7. Equally, it could still be that GCC 4.7 is performing an unsafe strict aliasing optimisation which it didn't do in 4.6. Weirdly the Boost bugtracker thinks the bug fixed in GCC 4.7, whereas this bug is about a 4.7 regression from 4.6. Niall
Maybe someone should look at fixing these warnings in Boost.Python, or ensure -fno-strict-aliasing is used "g++" -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -m64 -pthread -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_SOURCE -DNDEBUG -I"." -I"/apps/infra/3rd-party/gcc-4.4-64/python-2.6.5/include/python2.6" -c -o "bin.v2/libs/python/build/gcc-4.7.0/release/threading-multi/object/enum.o" "libs/python/src/object/enum.cpp" libs/python/src/object/enum.cpp: In function 'boost::python::api::object boost::python::objects::{anonymous}::new_enum_type(const char*, const char*)': libs/python/src/object/enum.cpp:150:11: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing] "g++" -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -m64 -pthread -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_SOURCE -DNDEBUG -I"." -I"/apps/infra/3rd-party/gcc-4.4-64/python-2.6.5/include/python2.6" -c -o "bin.v2/libs/python/build/gcc-4.7.0/release/threading-multi/object/class.o" "libs/python/src/object/class.cpp" libs/python/src/object/class.cpp: In function 'PyObject* boost::python::objects::static_data()': libs/python/src/object/class.cpp:211:11: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing] libs/python/src/object/class.cpp: In function 'boost::python::type_handle boost::python::objects::class_metatype()': libs/python/src/object/class.cpp:319:11: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing] libs/python/src/object/class.cpp: In function 'boost::python::type_handle boost::python::objects::class_type()': libs/python/src/object/class.cpp:473:11: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing] gcc.compile.c++ bin.v2/libs/python/build/gcc-4.7.0/release/threading-multi/object/life_support.o "g++" -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -m64 -pthread -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_SOURCE -DNDEBUG -I"." -I"/apps/infra/3rd-party/gcc-4.4-64/python-2.6.5/include/python2.6" -c -o "bin.v2/libs/python/build/gcc-4.7.0/release/threading-multi/object/life_support.o" "libs/python/src/object/life_support.cpp" libs/python/src/object/life_support.cpp: In function 'PyObject* boost::python::objects::make_nurse_and_patient(PyObject*, PyObject*)': libs/python/src/object/life_support.cpp:94:9: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing] libs/python/src/object/life_support.cpp:96:9: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing] gcc.compile.c++ bin.v2/libs/python/build/gcc-4.7.0/release/threading-multi/object/function.o "g++" -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -m64 -pthread -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_SOURCE -DNDEBUG -I"." -I"/apps/infra/3rd-party/gcc-4.4-64/python-2.6.5/include/python2.6" -c -o "bin.v2/libs/python/build/gcc-4.7.0/release/threading-multi/object/function.o" "libs/python/src/object/function.cpp" libs/python/src/object/function.cpp: In constructor 'boost::python::objects::function::function(const boost::python::objects::py_function&, const boost::python::detail::keyword*, unsigned int)': libs/python/src/object/function.cpp:108:9: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing] libs/python/src/object/function.cpp:110:9: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]
I think this is a duplicate of PR 53657, the crash happens returning a std::pair and goes away if the move ctor is trivial *** This bug has been marked as a duplicate of bug 53657 ***
(In reply to comment #0) > I tried boost as delivered with fedora 17, a home-compiled version with > -std=c++11 and a home-compiled version without c++11. The c++11 flag on the > _library_ does not seem to influence the problem, although the value change is > inside the library. However, when turning off c++11 for the testing program > (going to attach the .ii too), it works fine. Are you sure you built boost with -std=c++11 correctly? If I build libboost_python.so with -std=c++11 and then compile crashtest.cpp with -std=c++11 it works fine, likewise building both with -std=c++98. It only crashes when mixing c++98 library with c++11 code, or vice versa (which is consistent with PR 53657)
Agreed, but it is highly unlikely to happen anytime soon unless a new sponsor turns up. BPL needs a fair bit of post-bitrot work as it is. Niall (In reply to comment #12) > Maybe someone should look at fixing these warnings in Boost.Python, or ensure > -fno-strict-aliasing is used > > "g++" -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -m64 > -pthread -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_SOURCE -DNDEBUG -I"." > -I"/apps/infra/3rd-party/gcc-4.4-64/python-2.6.5/include/python2.6" -c -o > "bin.v2/libs/python/build/gcc-4.7.0/release/threading-multi/object/enum.o" > "libs/python/src/object/enum.cpp" > > libs/python/src/object/enum.cpp: In function 'boost::python::api::object > boost::python::objects::{anonymous}::new_enum_type(const char*, const char*)': > libs/python/src/object/enum.cpp:150:11: warning: dereferencing type-punned > pointer will break strict-aliasing rules [-Wstrict-aliasing] > > "g++" -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -m64 > -pthread -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_SOURCE -DNDEBUG -I"." > -I"/apps/infra/3rd-party/gcc-4.4-64/python-2.6.5/include/python2.6" -c -o > "bin.v2/libs/python/build/gcc-4.7.0/release/threading-multi/object/class.o" > "libs/python/src/object/class.cpp" > > libs/python/src/object/class.cpp: In function 'PyObject* > boost::python::objects::static_data()': > libs/python/src/object/class.cpp:211:11: warning: dereferencing type-punned > pointer will break strict-aliasing rules [-Wstrict-aliasing] > libs/python/src/object/class.cpp: In function 'boost::python::type_handle > boost::python::objects::class_metatype()': > libs/python/src/object/class.cpp:319:11: warning: dereferencing type-punned > pointer will break strict-aliasing rules [-Wstrict-aliasing] > libs/python/src/object/class.cpp: In function 'boost::python::type_handle > boost::python::objects::class_type()': > libs/python/src/object/class.cpp:473:11: warning: dereferencing type-punned > pointer will break strict-aliasing rules [-Wstrict-aliasing] > gcc.compile.c++ > bin.v2/libs/python/build/gcc-4.7.0/release/threading-multi/object/life_support.o > > "g++" -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -m64 > -pthread -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_SOURCE -DNDEBUG -I"." > -I"/apps/infra/3rd-party/gcc-4.4-64/python-2.6.5/include/python2.6" -c -o > "bin.v2/libs/python/build/gcc-4.7.0/release/threading-multi/object/life_support.o" > "libs/python/src/object/life_support.cpp" > > libs/python/src/object/life_support.cpp: In function 'PyObject* > boost::python::objects::make_nurse_and_patient(PyObject*, PyObject*)': > libs/python/src/object/life_support.cpp:94:9: warning: dereferencing > type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing] > libs/python/src/object/life_support.cpp:96:9: warning: dereferencing > type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing] > gcc.compile.c++ > bin.v2/libs/python/build/gcc-4.7.0/release/threading-multi/object/function.o > > "g++" -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -m64 > -pthread -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_PYTHON_SOURCE -DNDEBUG -I"." > -I"/apps/infra/3rd-party/gcc-4.4-64/python-2.6.5/include/python2.6" -c -o > "bin.v2/libs/python/build/gcc-4.7.0/release/threading-multi/object/function.o" > "libs/python/src/object/function.cpp" > > libs/python/src/object/function.cpp: In constructor > 'boost::python::objects::function::function(const > boost::python::objects::py_function&, const boost::python::detail::keyword*, > unsigned int)': > libs/python/src/object/function.cpp:108:9: warning: dereferencing type-punned > pointer will break strict-aliasing rules [-Wstrict-aliasing] > libs/python/src/object/function.cpp:110:9: warning: dereferencing type-punned > pointer will break strict-aliasing rules [-Wstrict-aliasing]
I think I built it correctly with std=c++11, but is there a way to verify this properly in the built library?
(In reply to comment #16) > I think I built it correctly with std=c++11, but is there a way to verify this > properly in the built library? crashtest.cpp doesn't crash ;)
(In reply to comment #17) > (In reply to comment #16) > > I think I built it correctly with std=c++11, but is there a way to verify this > > properly in the built library? > > crashtest.cpp doesn't crash ;) I think he means something like a magic weak symbol emitted to indicate a binary was built with C++11. If there isn't such a magic symbol yet, there should be. Niall
Right, because otherwise I would not consider that as a safe verification that this is indeed a duplicate of the referenced bug. And I like safe verifications.
That wouldn't help if you built one object with -std=c++11 and another with -std=c++98 and linked them both into the same .so, you'd have the symbol, but wouldn't have built everything with -std=c++11 Anyway, it's the same bug, it happens when passing a std::pair between c++11 code and c++98 code and if you make the std::pair move-ctor trivial then there's no crash.
So this boils down to that we cannot have a c++11/non-c++11 heterogenous environment on a system. One would have to build all libraries for both standards until c++11 is well established.
(In reply to comment #20) > That wouldn't help if you built one object with -std=c++11 and another with > -std=c++98 and linked them both into the same .so, you'd have the symbol, but > wouldn't have built everything with -std=c++11 > > Anyway, it's the same bug, it happens when passing a std::pair between c++11 > code and c++98 code and if you make the std::pair move-ctor trivial then > there's no crash. You miss my point: If built under C++03, a magic symbol __gplusplus_cplusplus_03 might be emitted; if built under C++11, a magic symbol __gplusplus_cplusplus_11 might be emitted. You can then tell if mixed build object files have been combined - and you can also tell if a given binary is pure. ld can then be patched to emit a warning if both symbols appear in the same binary. After all, mixing C++03 and C++11 is likely bad for your health forever into the future. Indeed, if it becomes a fatal problem for a lot of end users, gcc may need to mangle things specially for C++11 versus C++03 code for code so affected in order to prevent bad linking. The loss of std::pair interop between C++03 and C++11 in my mind is pretty fatal for a lot of end users. I can submit a wishlist issue for GCC for the above if it doesn't already exist? Niall
(In reply to comment #21) > So this boils down to that we cannot have a c++11/non-c++11 heterogenous > environment on a system. One would have to build all libraries for both > standards until c++11 is well established. See http://gcc.gnu.org/ml/gcc/2012-06/msg00201.html and http://gcc.gnu.org/wiki/Cxx11AbiCompatibility has some more details on incompatibilities. Bugzilla is the wrong place to discuss it, this PR is a dup of the std::pair issue.
(In reply to comment #22) > The loss of std::pair interop between > C++03 and C++11 in my mind is pretty fatal for a lot of end users. It's a bug. It's being addressed. > I can submit a wishlist issue for GCC for the above if it doesn't already > exist? Sure.
(In reply to comment #24) > (In reply to comment #22) > > I can submit a wishlist issue for GCC for the above if it doesn't already > > exist? > > Sure. Added as #53673. Niall