Security vulernarability or security feature?

[Robert Dewar]

James Courtier-Dutton wrote:
> 2008/4/25 Prateek Saxena <prateeksaxena@gmail.com>:
>> On Thu, Apr 24, 2008 at 2:20 PM, Ralph Loader <suckfish@ihug.co.nz> wrote:
>>  > > I am very interested in seeing how this optimization can remove
>>  >  > arithmetic overflows.
>>  >
>>  >  int foo (char * buf, int n)
>>  >  {
>>  >         // buf+n may overflow of the programmer incorrectly passes
>>  >         // a large value of n.  But recent versions of gcc optimise
>>  >         // to 'n < 100', removing the overflow.
>>  >         return buf + n < buf + 100;
>>  >  }
>>
>>  This clearly is insecure coding. The optimization to replace "buf + n
>>  < buf + 100" with "n < 100" assumes something about the value of buf.
>>  I assume that the location of "buf", could change arbitrarily accross
>>  platforms depending on the memory layout.
>>  I ran foo as follows, getting different outputs :
>>
> 
>>From the algebra perspective "buf + n < buf + 100" should be the same
> as "n < 100". There should really be a feature in C to handle the
> special overflow/carry case in order to handle overflows and wrap
> arounds.

This is not algebra with natural numbers, it is C, any programmer who
analyzes expressions expecting algebraic laws of this kind to hold is
in serious trouble!
> 
> This is probably a fundamental problem with C.
> If one takes the following:
> 
> int a,b,c;
> a = b + c;
> 
> How does one detect if there was a signed int overflow?

Not easily indeed

> Programmers are forced to jump all sorts of hoops to do this check.

Yes, this is a weakness of the language. The proper hoops in this
case are to do the addition in unsigned, and check the sign bit of
the result and operands. In cases where you know the operands are
positive, the check is of course much simpler.

> Fortunately there is a assembler instruction to do just this on most CPUs.
> e.g. jo, jc, js
> It would be nice to be able to write this sort of C code.
> 
> int a,b,c;
> a = b + c;
> if (a overflowed) {
>     handle_overflow();
> }

Yes, but there is no such feature
> 
> One cannot simply code a special function to test the carry/overflow
> flags as the compiler might reorder the previous instructions, and
> therefore the carry/overflow state gets messed up.

Right indeed.
You could code an addition function that did an overflow check and
call that function
> 
> Why has the C language lacked this sort of functionality for so long?

It would be a major language change, there are of course languages (e.g.
Ada) that handle this conveniently and properly. To me, the more 
interesting question is how on earth Java missed this fundamental
requirement.
> 
> James