Re: operator new[] overflow (PR 19351)

["Joseph S. Myers" <joseph at codesourcery dot com>]

On Tue, 30 Nov 2010, Florian Weimer wrote:

> Mozilla seems to receive a report of an exploitable operator new[]
> overflow every couple of months now.  Obviously, this is not good.
> 
> What is necessary so that GCC can fix this code generation issue?
> I've created a patch, together with a test case, but it has not been
> approved, nor have I been told how to change the patch to make it more
> suitable for inclusion ("change the middle end type system so that
> this can be expressed in a better way" is just not realistic for me,
> and apparently anyone else):
> 
>   <http://gcc.gnu.org/ml/gcc-patches/2010-02/msg00275.html>

My suggestion at <http://gcc.gnu.org/ml/gcc-patches/2010-02/msg00315.html> 
wasn't to change the type system; it was to add new GENERIC / GIMPLE codes 
within the existing type system.

Saturating arithmetic keeps coming up in one context or another.  It's 
generally relevant if you want to exploit various instructions on various 
processors - or if you want to support vector intrinsics (which may 
include saturating operations) even when the vector instructions aren't 
present (see recent discussions of doing that for SSE intrinsics, for 
example).  Sooner or later I expect someone will decide to do the work for 
at least one target.

Related to the "operator new[]" issue, I'm concerned that saturation alone 
may not be enough to solve problems with overflows in allocation.  If on a 
32-bit system you allocate 2GB or more of memory, then differences between 
addresses in / just after that array cannot be calculated reliably as they 
may excess the range of ptrdiff_t.  Thus actually malloc ought to reject 
allocations that take half or more of the address space, to avoid 
undefined behavior arising in the calling code (such allocations simply 
cannot be used reliably in C or C++).  But I don't see anything in glibc's 
malloc to do so; it seems quite possible that it could allocate over 2GB 
in a single allocation on a 32-bit system.  (See PR 45779, but I think the 
only sensible place to fix this is the implementation of malloc.)

-- 
Joseph S. Myers
joseph@codesourcery.com