[Bug c/20688] New: [x86-64] gcc miscompiles composition of sqrt() with inlined function

["cvogler2001 at yahoo dot com" <gcc-bugzilla at gcc dot gnu dot org>]

GCC info: gcc -v

Reading specs from
/home/cvogler/addons/gcc-test/lib/gcc/x86_64-unknown-linux-gnu/3.4.3/specs
Configured with: ../gcc-3.4.3/configure --prefix=/home/cvogler/addons/gcc-test
--disable-multilib
Thread model: posix
gcc version 3.4.3

The following test snippet shows a very reduced case of Cholesky factorization
code, which uses inline functions for array indexing. It is miscompiled with
these compiler flags on the x86_64 target:

gcc -Wall -O -finline-functions -o test test.c -lm

--- test.c ---
#include <math.h>

double getval()
{
  return -1.0;
}

int main(void)
{
  double y = sqrt(getval());
}
---

The result is in infinite loop, instead of the expected "nan." This generated
assembler snippet, in particular, looks incorrect:

---
.L3:
        sqrtsd  %xmm0, %xmm0
        ucomisd %xmm0, %xmm0
        jp      .L6
        je      .L4
.L6:
        movlpd  .LC3(%rip), %xmm0
        jmp     .L3
.L4:
        rep ; ret
---

In this snippet .L6 should contain a call to the sqrt function, but instead it
jumps back to .L3, causing an infinite loop. Suppressing inlining of functions
makes gcc generate correct code. Also, I could not reproduce the problem on a
32-bit x86 target.

-- 
           Summary: [x86-64] gcc miscompiles composition of sqrt() with
                    inlined function
           Product: gcc
           Version: 3.4.3
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: c
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: cvogler2001 at yahoo dot com
                CC: gcc-bugs at gcc dot gnu dot org
 GCC build triplet: x86_64-unknown-linux-gnu
  GCC host triplet: x86_64-unknown-linux-gnu
GCC target triplet: x86_64-unknown-linux-gnu


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=20688