Re: Problem regarding canonicalization

[Andrew Haley <aph at redhat dot com>]

Ashwin writes:
 > I have a combiner pattern that converts a sub-cmp pattern to a cmp insn, 
 > something like this -
 > "if (a-1 < 0)"
 > is converted to
 > "if (a<1)"
 > 
 > Now consider the following test case -
 > 
 > <snip>
 > f(long a){return (--a > 0);}
 > main(){if(f(0x80000000L)==0)abort();exit(0);}
 > </snip>
 > 
 > The compiler generates the following code for f()
 >        cmp     r0, 1   ;;canonicalized
 >        mov r0,1
 >        sub.le r0,r0,r0
 > 
 > This works fine under normal circumstances. However, in the testcase, 
 > the least negative no. i.e. 0x80000000 (hereafer referred to as MIN) is 
 > passed. When 1 is subtracted from MIN, by --a, it becomes positive and 
 > the conditions get reversed thus leading to failure during execution.
 > 
 > Similar problem seems to arise when MAX is passed to a function that 
 > does "return (++a < 0).
 > 
 > How do I tackle this problem? Anything I may be missing?

Yes.  Signed integer overflow is undefined in C, so this is not a
well-defined C program.  For example, this program may trap.  

So, the semantics of the C language allow this optimization.  See 6.5
Para 5.

For unsigned types C is much more specific, and you must not do any
optimizations that do not return the correct result modulo 2**N, where
N is the wordlength.  See 6.2.5 Para 9.

Some other languges targeted by gcc, such as the Java langauge, are
much more strict about the behaviour of integer overflow.

Andrew.