[Bug c++/11223] New: dynamic_cast<> in shared library fails with SegFault when called from JNI

[lemehmet@hotmail.com]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=11223

           Summary: dynamic_cast<> in shared library fails with SegFault
                    when called from JNI
           Product: gcc
           Version: 3.3
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: c++
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: lemehmet@hotmail.com
                CC: gcc-bugs@gcc.gnu.org
  GCC host triplet: Linux tramp 2.4.20-18.9 #1 Thu May 29 07:08:16 EDT 2003
                    i686 i68

Epilogue: I read the FAQ and aware of the problem.

As described in previous Bugs on dynamic_cast and the FAQ, using dynamic_cast<>
in a shared library which is called from JNI SegFaults. The problem is, JavaVM
is a black-box, thus its library loading mechanism cannot be changed. Indeed
this is one of the purpose of shared libraries existance: Binary extensibility. 

The DoIt() function in the pasted code below always generates SegFault in the
JNI case.

Test Cases:
1. Load with:
   void *pLib = dlopen("libTestus.so", RTLD_NOW | RTLD_GLOBAL);
   Runs OK.

2. Load with:
   void *pLib = dlopen("libTestus.so", RTLD_LAZY);
   Runs OK.
   After reading the FAQ, one would expect that this should SegFault.

3. Dynamic Link Edit.
   Runs OK.

4. Load within Java and invoke via JNI.
   SegFaults.

If dlopen(RTLD_LAZY) were also failed, one could assert that Java VM loads the
library without RTLD_GLOBAL that's why dynamic_cast fails as described in the
FAQ. This does not seem to be happening.
dynamic_cast never fails if called from C++ code, even though -Wl,-E is not used
during linkage of shared library.
But always fails if loaded and called from Java VM.

Obviously this fault prevents C++ development for JNI usage, which renders gcc
unusable for JNI.

The same code works on these platforms without any problems:
Microsoft Windows XP SP1, Microsoft Visual Studio 6 SP5
Sun Solaris 8/9, Sun Forte Developer 7 C/++ 5.4, JavaVM 1.4.1_02-b06
Fails on this platform:
Linux tramp 2.4.20-18.9, gcc 3.3, Java VM 1.4.1_02-b06

I will attach a test case as soon as I find where I can.

---code------code------code---

class A
{
public:
    virtual void Exec()
    {
        printf("A::Exec()\n");
    }
};

class B : public A
{
public:
    virtual void Exec()
    {
        printf("B:Exec()\n");
        A::Exec();
    }
};

class C : public A
{
public:
    virtual void Exec()
    {
        printf("C:Exec()\n");
        A::Exec();
    }
};

extern "C" void DoIt()
{
    printf("Doing it.\n");
    A *pA = new A();
    A *pB = new B();
    A *pC = new C();
    
    printf("Casting from A(B) to B...");
    B *pCastedB = dynamic_cast<B*>(pB);
    if (pCastedB == NULL)
    {
        printf("Failed.\n");
    }
    else
    {
        printf("Done\n");
    }
    
    printf("Casting from A(C) to C...");
    C *pCastedC = dynamic_cast<C*>(pC);
    if (pCastedC == NULL)
    {
        printf("Failed.\n");
    }
    else
    {
        printf("Done\n");
    }

    printf("Casting from A(C) to B...");
    B *pWronglyCastedB = dynamic_cast<B*>(pC);
    if (pWronglyCastedB == NULL)
    {
        printf("Failed as expected.");
    }
    else
    {
        printf("Should have failed to apply dynamic_cast, but succeeded.\n");
    }
}

---code------code------code---