When building 64 bit with an optimization setting of -O2, -O3, or -Os, some binary comparisons with nan values fail to produce false. All comparisons (==, >, <, etc) where one (or both) of the two values is nan should produce false.
There are no warnings or other error messages during compilation. The attached test demonstrates the error with the final assert failing. The max() function should return the maximum of the two values, ignoring nans. i.e., max(5, nan) should be 5. This fails with gcc3.4.6 -O2 -m64.
Strangely, switching the order of the final portion of the return to something like:
return (a > b ? a : (b > a ? b : (isNaN(b) ? a : b)));
causes it to work.
Also, replacing isNaN() with a call to the slower, non-inline isnan() also makes the function work.
Created attachment 13353 [details]
Are you sure you did not have -ffast-math on the command line?
-ffast-math was not used. Adding that option actually makes the test work.