Obfuscate the vtpr inside polymorphic objects
Jonathan Wakely
jwakely.gcc@gmail.com
Sun Nov 9 21:12:31 GMT 2025
On Sun, 9 Nov 2025, 14:51 Frederick Virchanza Gotham via Gcc, <
gcc@gcc.gnu.org> wrote:
> Hi guys
>
> I'm kind of new to editing compilers but I'm enthusiastic to learn.
> I'm currently working on the following patch to the GNU g++ compiler:
>
>
> https://github.com/gcc-mirror/gcc/compare/trunk...healytpk:gcc-vptr-xor:trunk
>
> For polymorphic objects in C++, I want to XOR the vptr's value with
> its own address, meaning that if you relocate a polymorphic object,
> you need to re-encode the vptr, otherwise you'll get a segfault when
> you go to invoke a virtual method. I am doing this in order to
> simulate what happens on Apple Silicon computers with the arm64e
> architecture with pointer authentication, i.e. to prove the need for a
> 'std::restart_lifetime' function in the C++ standard library.
>
> I was hoping some of you guys could help me with my patch. For
> instance, I don't know how to XOR the vptr when the variable is either
> constinit or constexpr, and this is why I do the following inside the
> file "gcc/cp/class.cc":
>
> if (classtype && TYPE_HAS_CONSTEXPR_CTOR (classtype))
> return vtbl; /* For classes with constexpr ctors, never
> encode the vptr. */
>
> Do you think might it be possible to also XOR the vptr's of
> consinit/constexpr objects?
>
> Also my implementation of "__restart_lifetime" crashes . . . I'm not
> sure which enum I'm supposed to edit, "builtin_function" or
> "cp_builtin_function". Perhaps could someone do a quick scroll through
> my patch above and give me some pointers?
>
> Below is the email I sent earlier today to the C++ standard proposals
> mailing list:
> - - - - - - - - - -
> At the Kona talk on Wednesday evening just gone, we talked about
> relocation.
>
> We went into the complications of the 'arm64e' architecture which has
> pointer authentication. Most of us on this mailing list have an x86_64
> computer, and so I wanted to put together a test suite that doesn't
> require a new Apple Silicon computer.
>
> I have edited the GNU g++ compiler to obfuscate the vtable pointer
> inside a polymorphic object. I have it tested and working on x86_64.
> From the looks of things, I think it will work on every CPU and
> operating system that g++ can run on.
>
> I've changed g++ so that when it writes a vptr to a polymorphic
> object, it XOR's the address of the vtable with the address of the
> vptr. Here's my compiler patch:
>
>
> https://github.com/gcc-mirror/gcc/compare/trunk...healytpk:gcc-vptr-xor:trunk
>
> Just one thing: If the polymorphic class has one or more
> constexpr/consteval constructors, I don't obfuscate the vptr -- this
> is because I couldn't figure out how to do it without getting an ICE
> error. So when testing, make sure the class hasn't got a
> constexpr/consteval constructor. When writing the vtpr to the
> polymorphic object, I set the least significant bit to 1 as a flag to
> indicate that it has been XOR'ed (which is fine because this bit will
> always be zero -- even after the XOR).
>
> I built this compiler and then got it to compile the following source file:
>
> #include <cstdio> // puts
>
> struct Monkey {
> int n;
> Monkey(int const arg) : n(arg) {}
> virtual void Func(void);
> };
>
> void Monkey::Func(void)
> {
> std::puts("Hello World");
> }
>
> void Invoke(Monkey &m)
> {
> m.Func();
> }
>
> Before I show you how the new compiler assembled the function
> 'Invoke', let's take a look at how a normal g++ compiler does it:
>
> Invoke:
> mov (%rdi), %rax
> jmp *(%rax)
>
> The first instruction dereferences the object pointer to get the vptr.
> The second instruction dereferences the vptr to get the address of the
> first virtual function, and then jumps to the first virtual function's
> machine code.
>
> So with the new compiler, we're expecting to see the vptr XOR'ed with
> its own address before it's dereferenced . . . okay so let's see the
> output from 'objdump -d source.o' for the new compiler:
>
> Invoke:
> mov (%rdi), %rax ; load vptr (possibly encoded)
> test $0x1, %al ; test low bit of vptr (tag bit: 1 =
> encoded, 0 = plain)
> je .Lplain ; if low bit is 0 -> not encoded,
> jump to plain case
>
> and $0xfffffffffffffffe, %rax ; clear low tag bit (RAX &= ~1)
> xor %rdi, %rax ; decode vptr: RAX = RAX ^ this (this
> is in RDI)
> mov (%rax), %rax ; load function pointer from decoded
> vtable[0] into RAX
> cmp $0x0, %rax ; compare function pointer to null
> (sanity check)
> jne .Lcall ; if non-null, go call it
> ret ; if null, just return
>
> nopw 0x0(%rax,%rax,1) ; 6-byte NOP (padding/alignment)
>
> .Lplain:
> mov (%rax), %rax ; plain vptr case: original vptr is a
> real vtable ptr; load vtable[0]
> cmp $0x0, %rax ; compare function pointer to null
> je .Lret ; if null, return
>
> .Lcall:
> jmp *%rax ; tail-call via the function pointer
> (virtual Monkey::Func)
>
> .Lret:
> ret
>
> This is mildly cool. We've got some sort of basic pointer validation
> going on now, and we can test it natively on x86_64 computers running
> Linux or MS-Windows or macOS or FreeBSD. I think it should work on
> every computer for which you can build the GNU compiler.
>
> The following program works fine when compiled with the normal g++
> compiler, but it crashes with the new compiler:
>
> #include <cstring> // memcpy
>
> struct Monkey {
> int n;
> Monkey(int const arg) : n(arg) {}
> virtual void Func(void);
> };
>
> extern void Invoke(Monkey&);
>
> int main(int const argc, char **const argv)
> {
> Monkey m(argc);
> alignas(Monkey) char unsigned buf[ sizeof(Monkey) ];
> std::memcpy( &buf, &m, sizeof buf );
> Monkey &m2 = *static_cast<Monkey*>( static_cast<void*>( &buf ) );
> Invoke(m2);
> }
>
> It crashes because the vptr is now corrupt after relocation. The
> solution is to use "std::restart_lifetime". In the new compiler, I
> have added a new built-in function called '__builtin_restart_lifetime'
> but I haven't got it working quite perfectly yet, and so for the time
> being I have edited the standard library header file <memory> and
> given it a rudimentary naive implementation of 'std::restart_lifetime'
> as follows:
>
> template <typename _Tp>
> _Tp*
> restart_lifetime(const _Tp* const __old_p, _Tp* const __new_p)
> {
> // I have only used C++11 features in this implementation.
>
> // This is a naive implementation that will only XOR the vptr
> // of the most-derived object -- it won't XOR the vptr inside
> // sub-objects, and so will only work properly for simple classes.
> // This implementation is only intended for testing until the
> // compiler gets a built-in function to do this properly, e.g.
> // __restart_lifetime
>
> static_assert( false == is_const<_Tp>::value, "T must not be
> const" );
> if ( false == is_polymorphic<_Tp>::value ) return __new_p;
> uintptr_t volatile &n = *static_cast<uintptr_t volatile*>(
> static_cast<void volatile*>( __new_p ) );
> if ( 0u == (n & 1u) ) return __new_p;
> n ^= reinterpret_cast<uintptr_t>( __old_p );
> n ^= reinterpret_cast<uintptr_t>( __new_p );
> return __new_p;
> }
>
> So let's go back to our program that was crashing, and let's try add
> one line of code to get it to work:
>
> int main(int const argc, char **const argv)
> {
> Monkey m(argc);
> alignas(Monkey) char unsigned buf[ sizeof(Monkey) ];
> std::memcpy( &buf, &m, sizeof buf );
> Monkey &m2 = *static_cast<Monkey*>( static_cast<void*>( &buf ) );
> ++++ std::restart_lifetime(&m,&m2);
> Invoke(m2);
> }
>
> With this new line invoking restart_lifetime, the program no longer
> crashes :-)
>
> So now we have a working compiler for x86_64 that we can use for
> testing relocation of polymorphic objects that have an obfuscated
> vptr. I wonder if Matt will put this up on GodBolt. . . . I'll email
> him now.
>
You need to send a pull request to add it yourself, not just ask Matt or
the other CE admins to do it for you. They're all busy people.
> I do realise the compiler needs more polishing and that I have to fix
> __builtin_restart_lifetime. Plus if anyone reading this has
> compiler-writing experience and knows how to make this work with
> consinit/constexpr variables, I am of course all ears.
>
More information about the Gcc
mailing list