c/2684: gcc 2.95.3 bad optimization corrupts x86 FP stack

[mat at curl dot com]

>Number:         2684
>Category:       c
>Synopsis:       gcc 2.95.3 bad optimization corrupts x86 FP stack
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    unassigned
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Apr 29 10:06:00 PDT 2001
>Closed-Date:
>Last-Modified:
>Originator:     Mat Hostetter
>Release:        gcc 2.95.3
>Organization:
>Environment:
x86/linux
>Description:
poison-ivy:~$ gcc -v
Reading specs from /usr/local/lib/gcc-lib/i686-pc-linux-gnu/2.95.2/specs
gcc version 2.95.2 19991024 (release)

I am told this happens in gcc-2.95.3 as well.  I am told that it
doesn't happen in RedHat's gcc-2.96-71 RPM, but the bug is
sufficiently tricky to reproduce that its success might be luck.
I'm filing this bug to make sure it is known to be fixed
"for real".

Anyway, here's my problem description:

The x86 has an 8-deep floating point stack.  gcc is required to push
and pop the same number of things on the stack, because when that
stack overflows, you start getting NaNs.

gcc -O2 (not gcc -O) has a bad optimization that causes a function to
leave cruft on the FP stack.  Here's my guess for what's happening,
based on looking at the assembly code produced:

gcc pushes some values on the FP stack so it can compare them.  But
then, as an optimization, it realizes that the "then" and "else"
clauses of the "if" are identical, so it doesn't bother generating the
compare instruction.  That's a legitimate optimization, but because it
doesn't do the compare, gcc never gets around to popping the values
off the FP stack.

This bug shows the FP stack overflowing.  The code is ugly because
it's derived from machine-generated code.  Anyway, if you run it
you'll see this as the result of 0.5 + 0.5:

poison-ivy:~$ gcc -O2 -Wall -Werror -I /u/mat/compiler bad.c -o bad && ./bad 
  1.000000
  1.000000
  1.000000
  nan
  nan
  nan
  nan
  nan
  nan
  nan




void
get_nums (double d[3])
{
  d[0] = 1;
  d[1] = 2;
  d[2] = 3;
}

void
leave_cruft_on_fpu_stack (double d[2])
{
  double ret[3];
  double f0, f1, f2, f3, f4;
  int t13;

  get_nums (ret);
  f0 = ret[0];
  f1 = ret[1];
  f2 = ret[2];
  f3 = f0 / f2;
  f4 = f1 / f2;
  t13 = printf (" ");
  printf (" ");
  if (f3 == 1.0) goto L1;
  if (f3 == 0.0) goto L1;
  if (f4 == 1.0) goto L1;
  goto L3;
L1:;
  d[0] = f3;
  goto L4;
L3:;
  d[0] = f3;
L4:;
}

volatile double n = 0.5;

int
main (int argc, char *argv[])
{
  double d[2];
  int i;

  for (i = 0; i < 10; i++)
    {
      leave_cruft_on_fpu_stack (d);
      printf ("%f\n", n + n);
    }

  return 0;
}
>How-To-Repeat:
Compile and run the given program with gcc -O2	
>Fix:

>Release-Note:
>Audit-Trail:
>Unformatted: