This is the mail archive of the mailing list for the GCC project.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Integer overflow in operator new

Hi all, apologies if this has been discussed before, but I
couldn't find anything about this issue in gcc mailing list

Use of operator new (et al) appears to have an integer overflow;
this function:
    int * allocate_int(size_t n)
        return new int[n];

with gcc-4.1 on IA-32 compiles to:

            pushl   %ebp
            movl    %esp, %ebp
            subl    $8, %esp
            movl    8(%ebp), %eax
            sall    $2, %eax       ####
            movl    %eax, (%esp)
            call    _Znaj

which is equivalent to the compilation of:

    int * allocate_int(size_t n)
        return (int*) operator new[](4 * n);

"4 * n", unchecked, is vulnerable to integer overflow.  On IA-32,
"new int[0x40000001]" becomes equivalent to "new int[1]".  I've
verified this on gcc-2.95 through 4.1.  For larger objects the
effects are exaggerated; smaller counts are needed to overflow.

This is similar to the calloc integer overflow vulnerability in
glibc, which was fixed back in 2002.  Interestingly, RUS-CERT
2002-08:02 did mention 'operator new', and so did Bugtraq 5398.

See also this 2004 article by Raymond Chen:

Possible fixes might be to abort or to pass ULONG_MAX (0xFFFFFFFF)
to 'operator new' and let it return NULL or throw bad_alloc.

At least one other compiler already specifically guards against
integer overflow in 'operator new'.

Karl 2007-04-06 07:30

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]