This bug applies at least to gcc version 4.0.3 and the current CVS 4.2.0 20061103. The assembly extracts included below were produced by gcc 4.2.0; the gcc 4.0.3 extracts are analogous with respect to this problem but are different in some respects. My particular host is an intel pentium4 xenon.

With certain math optimizations enabled, gcc assumes that a test for isnan will always fail. This assumption is not correct, and results in the undesired propagation of NANs into other computations and altered control flow.

This bug is exemplified by the following code which can be compiled without including any header files. For brevity in the test case, instead of include files I have used only the prototypes needed to silence gcc's warnings. The code compiles the same, when <math.h> and <stdio.h> are included instead of the prototypes, or when the prototypes are simply left out all together.

float logf(float);
int printf(const char *fmt, ...);
int isnan();

double test(int i0, int n, double a) {
 double sum = 0.0;
 int i;

 for(i=i0; i<n; ++i) {
     float x = logf((float)i);
     sum += isnan(x) ? 0 : x;
 }

 return sum;
}

int main(void) {
        printf("test(4, 6, 0) = %f\n", test(4,6,0));
        printf("test(0, 2, 0) = %f\n", test(0,2,0));
        printf("test(-2, 3, 0) = %f\n", test(-2,3,0));
        return 0;
}


james@recoil:~/project/cf/util$ /home/james/local/gcc/bin/gcc -O2
-march=i686 -funsafe-math-optimizations -fno-math-errno test.c -o
test

james@recoil:~/project/cf/util$ ./test
test(4, 6, 0) = 2.995732
test(0, 2, 0) = -inf
test(-2, 3, 0) = nan  <- incorrect, as written, test should preclude nan as a result

james@recoil:~/project/cf/util$ /home/james/local/gcc/bin/gcc -O2
-march=i686 test.c -o testb -lm

james@recoil:~/project/cf/util$ ./testb
test(4, 6, 0) = 2.995732
test(0, 2, 0) = -inf
test(-2, 3, 0) = -inf  <- correct

james@recoil:~/project/cf/util$ /home/james/local/gcc/bin/gcc -v
Using built-in specs.
Target: i686-pc-linux-gnu
Configured with: ../gcc-4-2/configure --prefix=/home/james/local/gcc
Thread model: posix
gcc version 4.2.0 20061103 (prerelease)

The assembly produced for the test case that runs correctly has this as the main loop:

 .L5:
         pushl   %ebx
         fildl   (%esp)
         addl    $4, %esp
         fstps   (%esp)
         fstpl   -24(%ebp)
         call    logf
         fucomi  %st(0), %st   ;; check for NAN
         fldz
         fcmovnu %st(1), %st   ;; conditional based on check
         fstp    %st(1)
         addl    $1, %ebx
         cmpl    %esi, %ebx
         fldl    -24(%ebp)
         faddp   %st, %st(1)
         jne     .L5

The (over) optimized assembly has this for the loop:

.L5:
       pushl   %eax
       addl    $1, %eax
       fildl   (%esp)
       addl    $4, %esp
       cmpl    %edx, %eax
       fldln2
       fxch    %st(1)
       fyl2x     ;; log is computed inline due to -funsafe-math-optimizations
       faddp   %st, %st(1) ;; result is added unconditionally this does
                           ;; not belong in -funsafe-math-optimizations
                           ;; (perhaps -funusable-math-optimizations)
       jne     .L5

Uros Bizjak reported (on the gcc mailing list) that 
(1) gcc 4.3 does not exhibit this incorrect behavior
(2) this is a bug.