Re: Signed int overflow behavior in the security context

[Paul Schlie <schlie at comcast dot net>]

> David Daney wrote:
>> Paul Schlie wrote:
>> (however as you appear to be describing an algorithm attempting to rely on
>> the implicit addresses of object storage locations resulting from an assumed
>> calling or allocation convention; and as such assumptions are well beyond
>> the scope of most typical language specifications; it' not clear that such
>> an algorithm should ever be presumed to reliably work regardless of any
>> applied optimizations?)
>
> Isn't that the gist of the entire overflow wraps issue? Signed overflow is
> undefined in C and always has been. It' not clear that any program that relies
> on it should ever be presumed to reliably work regardless of any applied
> optimization.

Almost, if implementations had historically initialized variable memory
to some value and programmers had correspondingly relied on that behavior
although the language specified it as being undefined, then I would
correspondingly argue that although the compiler may assign a variable
to a register, it should correspondingly initialize that register with the
value historically expected as if that value were stored in memory.

However as compilers and machines have not historically initialized memory
in this way, there is no historical semantic to preserve as exists with
signed integer overflow semantics, which have been well understood to wrap
on most all implementations regardless of the languages non-guarantee of
this behavior for good or bad.

Personally although I prefer wrapping semantics, I think consistently in the
presence of optimization is more important, and thereby would prefer an
implementation which enforces trap-on-overflow (hopefully recoverably so, as
forcibly terminating a program in the presence of an error without any means
to transparently recover is about the worst behavior imaginable), in lieu of
an schizophrenic implementation which changes the languages observable
semantics during optimization.