This is the mail archive of the gcc@gcc.gnu.org mailing list for the GCC project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Integer overflow in operator new


Hi all, apologies if this has been discussed before, but I
couldn't find anything about this issue in gcc mailing list
archives.

Use of operator new (et al) appears to have an integer overflow;
this function:
     
    int * allocate_int(size_t n)
    {
        return new int[n];
    }

with gcc-4.1 on IA-32 compiles to:

    _Z12allocate_intj:
            pushl   %ebp
            movl    %esp, %ebp
            subl    $8, %esp
            movl    8(%ebp), %eax
            sall    $2, %eax       ####
            movl    %eax, (%esp)
            call    _Znaj
            leave
            ret

which is equivalent to the compilation of:

    int * allocate_int(size_t n)
    {
        return (int*) operator new[](4 * n);
    }

"4 * n", unchecked, is vulnerable to integer overflow.  On IA-32,
"new int[0x40000001]" becomes equivalent to "new int[1]".  I've
verified this on gcc-2.95 through 4.1.  For larger objects the
effects are exaggerated; smaller counts are needed to overflow.

This is similar to the calloc integer overflow vulnerability in
glibc, which was fixed back in 2002.  Interestingly, RUS-CERT
2002-08:02 did mention 'operator new', and so did Bugtraq 5398.
http://cert.uni-stuttgart.de/advisories/calloc.php
http://www.securityfocus.com/bid/5398/discuss

See also this 2004 article by Raymond Chen:
http://blogs.msdn.com/oldnewthing/archive/2004/01/29/64389.aspx

Possible fixes might be to abort or to pass ULONG_MAX (0xFFFFFFFF)
to 'operator new' and let it return NULL or throw bad_alloc.

At least one other compiler already specifically guards against
integer overflow in 'operator new'.

-- 
Karl 2007-04-06 07:30


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]