Re: Is it OK that gcc optimizes away overflow check?

[Ian Lance Taylor <iant at google dot com>]

Agner Fog <agner@agner.org> writes:

> On 25-07-2011 08:04, Ian Lance Taylor wrote:
>> There are arguments on both sides of an issue like whether a
>> compiler should optimize based on strict overflow. When facing
>> arguments on both sides, which should we pick? When possible and
>> feasible, we pick the alternative which is written in the
>> standard. That seems to me to be the most reasonable solution to
>> such a problem. 
> My point is that you are over-interpreting the standard when you
> conclude that the compiler is allowed to do anything in case of
> overflow.

I don't think so.  gcc's treatment of signed overflow is the same as its
treatment of a number of other aspects of the standard.  The standard
says that signed overflow is undefined behaviour.  The standard says
that a valid program may not rely on undefined behaviour.  Therefore,
gcc can reasonably conclude that signed overflow can not occur.


>> It's reasonably straightforward to check for overflow of any operation
>> by doing the arithmetic in unsigned types.  By definition of the
>> language standard, unsigned types wrap rather than overflow.
> This is still optimized away without warning:
>> #include <stdlib.h>
>>
>> int func(int x) {
>>    int y = abs(x);
>>    if ((unsigned int)y > ~(0u) >> 1)  y = 123;
>>    return y;
>> }

If you are trying to test whether abs(x) overflows, then that is not the
right test.  You have to test before the operation, not after.


> Unsigned and signed don't overflow at the same point. There is no
> straightforward way you can convert the overflow of the abs() function
> to an unsigned wraparound.
>
> Is this what you call reasonably straightforward:
>>    int x, y;
>>    if ((unsigned int)x == ~(~0u >> 1))  { /* deal with overflow */}
>>    else y = abs(x);
> The code will become ugly and unreadable if you fill it with checks
> like this. And it still relies on the 2's complement, which is not
> guaranteed by the standard.

This should work reliably:

#include <stdlib.h>
int func(int x) {
  unsigned int y = (unsigned int) x;
  if (y == -y)
    return 123;
  return abs(x);
}


> You don't know that you need to be security conscious until it is too
> late :-)

That could well be true.  Nevertheless, it does not follow that gcc
should assume that you know what you want.  I have come around to
believing that the first step in being security conscious is to not use
C/C++.  They can be used in a fully secure manner by experts.  However,
you are concerned about non-experts, and I think that history has
demonstrated clearly that C/C++ can not be used securely by non-experts.
Signed overflow is just one tiny aspect of security problems caused by
C/C++.


>>> 3). I think that you are interpreting the C/C++ standard in an
>>> over-pedantic way. There are good reasons why the standard says that
>>> the behavior in case of integer overflow is undefined. 2's complement
>>> wrap-around is not the only possible behavior in case of
>>> overflow. Other possibilities are: saturate, signed-magnitude
>>> wrap-around, reserve a bit pattern for overflow, throw an
>>> exception. If a future implementation uses internal floating point
>>> representation for integers then an overflow might variously cause
>>> loss of precision, INF, NAN, or throw an exception. I guess this is
>>> what is meant when the standard says the behavior is undefined. What
>>> the gcc compiler is doing is practically denying the existence of
>>> overflow (
>>> http://www.mail-archive.com/pgsql-hackers@postgresql.org/msg105239.html
>>> ) to the point where it can optimize away an explicit check for
>>> overflow. I refuse to believe that this is what the standard-writers
>>> intended. There must be a sensible compromize that allows the
>>> optimizer to make certain assumptions that rely on overflow not
>>> occurring without going to the extreme of optimizing away an overflow
>>> check.
>> It would be interesting to try to write such a compromise.
> I think it would be more sound to use pragmas than command line
> options. A pragma could be placed precisely at the place in the code
> where there is a problem, telling whether overflow should be ignored
> or not. If you apply a command line option to a specific module
> somewhere in the makefile of a big project, other people working on
> the same project would not know why it is there and it could easily be
> messed up when the project is restructured.

__attribute__ ((optimize ("-fno-strict-overflow"))) should work already
at the function level.  Some work has been done toward making _Pragma
work at the statement or expression level, but it's hard to implement in
gcc's framework.  That will be the way to go going forward, I think.


> The compiler could either use the safe options by default and produce
> warning messages at missed optimization opportunities, or use unsafe
> options by default and produce warning messages when it makes unsafe
> optimizations.

Or the compiler could use the language standard by default, and produce
warning messages upon request when it makes potentially unsafe
optimizations.  That is what we are already doing today.

Ian