[Bug optimization/13608] [3.3 regression] Incorrect code with -O3 -ffast-math

[roger at eyesopen dot com]

------- Additional Comments From roger at eyesopen dot com  2004-01-08 17:11 -------
I beleive I've managed to trace the location of the bug in GCC.  The minimal
change to the mark.cpp testcase to change it from functioning to non-functioning
is to change the return type of MaxZDist2 from float to double.  This "delta"
then allowed me to compare the assembly language output between the two versions.

After many hours of eliminating label numbering and register allocation
differences, I managed to reduce the differences between the two .s files
to just:

162c162,164
<       fstps   -28(%ebp)
---
>       fstps   -40(%ebp)
>       movl    -40(%ebp), %eax
>       movl    %eax, -28(%ebp)

Where the second version aborts and the first version doesn't!  I
believe that the problem is that -40(%ebp) isn't a valid temporary
stack slot, which ends up corrupting the stack.  Changing -40(%ebp)
to -28(%ebp) resolves the failure (so it isn't an IA-32 RAW hazard :)

The issue is that the function MaxZDist2 looks like:

float MaxZDist2(...)
{
  if (!n) return 0.0f;
  ...
  return max_dist2;
}

Which is being transformed (in the broken float form) into the sequence:

        ...
        movl $0x0, %eax     // float return value allocated to %eax 
        if (!n) goto L21
        ...
        fstps   -40(%ebp)   // calculated max_dist2 calculated in st(0)
        movl    -40(%ebp), %eax  // move max_dist2 into return value
.L21:
        movl    %eax, -28(%ebp)  // store result of MaxZDist2

The "double" version which works perfectly, instead uses the sequence

        ...
        fldz
        if (!n) goto L21
        fstp %st(0)
        ...
.L21:
        fstps -28(%ebp)

where the contents of the ... and the rest of the function are identical.

The question now arises of how does GCC manage to use an invalid stack
slot, -40(%ebp)???  I suspect that the interaction is caused by the fact
that MaxZDist2 is being inlined into FindBestRotation (-finline-functions
or -O3 is necessary), and that FindBestRotation is calling alloca before
MaxZDist2 is called (changing the alloca to malloc or auto variables makes
the problem go away).  Perhaps prior to inlining -40(%ebp) was known to
be the top of the stack, but this was never corrected for upon inlining.
Certainly, the function prologues and epilogues are identical between the
working (double) and broken (float) forms, even though the float version
clearly uses an additional stack slot that the double version doesn't.

I've also confirmed that GNU gcc 3.3.1 is affected but that GNU gcc 3.3
isn't.  The problem was originally reported using the Fedora system compiler,
but I've failed to reproduce it with the system gcc-3.3.1 that comes with
SuSe 9.0.


-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |NEW
     Ever Confirmed|                            |1
   Last reconfirmed|0000-00-00 00:00:00         |2004-01-08 17:11:21
               date|                            |


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=13608