operator new[] overflow (PR 19351)

[Gabriel Dos Reis]

On Wed, Dec 1, 2010 at 5:36 PM, Chris Lattner <clattner@apple.com> wrote:
>
> On Nov 30, 2010, at 3:12 PM, Joe Buck wrote:
>
>> On Tue, Nov 30, 2010 at 01:49:23PM -0800, Gabriel Dos Reis wrote:
>>> The existing GCC behaviour is a bit more perverse than the
>>> C malloc() case as in
>>>
>>>       new T[n]
>>>
>>> there is no multiplication that could be credited to careless programmer.
>>> The multiplication is introduced by GCC.
>>
>> ... which suggests strongly that GCC should fix it.  Too bad the ABI is
>> frozen; if the internal ABI kept the two values (the size of the type, and
>> the number of values) separate and passed two arguments to the allocation
>> function, it would be easy to do the right thing (through bad_alloc if the
>> multiplication overflows).
>
> You don't need any ABI changes to support this.  For example, clang compiles:
>
> int *foo(long X) {
>  return new int[X];
> }
>
> into:
>
> __Z3fool:                               ## @_Z3fool
> Leh_func_begin0:
> ## BB#0:                                ## %entry
>        movl    $4, %ecx
>        movq    %rdi, %rax
>        mulq    %rcx
>        testq   %rdx, %rdx
>        movq    $-1, %rdi
>        cmoveq  %rax, %rdi
>        jmp     __Znam
>
> On overflow it just forces the size passed in to operator new to -1ULL, which throws bad_alloc.

This is a very good point.  At the minimum, GCC should generate similar code
if not improve on it.

-- Gaby