Re: Signed int overflow behavior in the security context

[Paul Schlie <schlie at comcast dot net>]

>> On Fri, Jan 26, 2007 at 06:57:43PM -0500, Paul Schlie wrote:
>> > Robert Dewar wrote:
>> >
>> > People always say this, but they don't really realize what they are
>> > saying. This would mean you could not put variables in registers, and
>> > would essentially totally disable optimization.
>> 
>> - can you provide an example of a single threaded program where the
>> assignment of variable to a machine register validly changes its
>> observable logical results?
>
> If the program has a hash table that stores pointers to objects, and
> the hash function depends on pointer addresses, then the choice to
> allocate some objects in registers rather than in stack frames will change
> the addresses. If the algorithm depends on the order of hash traversal,
> then -O2 will change its behavior.

- if the compiler chooses to alias an object's logical storage location
utilizing a register, and that object's logical address is well specified
by a pointer whose value is itself subsequently utilized; it shouldn't have
any logical effect on that object's logical pointer's value; as it's the
responsibility of the compiler to preserve the semantics specified by the
program.

(however as you appear to be describing an algorithm attempting to rely on
the implicit addresses of object storage locations resulting from an assumed
calling or allocation convention; and as such assumptions are well beyond
the scope of most typical language specifications; it' not clear that such
an algorithm should ever be presumed to reliably work regardless of any
applied optimizations?)

> Likewise, if the program has an uninitialized variable, the behavior
> will differ depending on details of optimization and how variables are
> assigned to memory.  Heap allocated for the first time might always be
> zero (as the OS delivers it that way), turning on optimization might then
> result in a nonzero initial value because of reuse of a register.

- I would argue that in this circumstance although the resulting value may
differ, the results are actually equivalent; as in both circumstances the
value returned is the value associated with it's storage location; and as
the value of all storage locations are arbitrary unless otherwise well
specified; the result may change from run to run regardless of any applied
optimizations.