__builtin_isnanl() and invalid x87 80-bit floating point numbers

[Liu Hao]

On 2017/5/12 18:48, Vincent Lefevre wrote:
> It seems to be regarded as a NaN, not as a trap representation. On
> Debian/unstable with an Intel(R) Xeon(R) CPU E5-2609 v3 (x86_64):
> 
> ------------------------------------------------------------------
> #include <stdio.h>
> 
> int main (void)
> {
>    long double ld = 0;
>    unsigned char data[10] = {
>      0x5b, 0x01, 0x04, 0x5e, 0x85, 0x00, 0x00, 0x00, 0xd8, 0x59
>    };
>    __builtin_memcpy(&ld, data, 10);
>    ld += 1;
>    printf ("%Lg\n", ld);
>    return 0;
> }
> ------------------------------------------------------------------
> 
> cventin:~> gcc-4.9 tst.c -o tst
> cventin:~> ./tst
> -nan
> 
That is because floating point exceptions are masked by default. The 
trap can be observed by unmasking the IM mask in x87 control word:

----------------------------------------
lh_mouse@lhmouse-dev:~$ cat test.c
#include <stdio.h>

int main (void)
{
    unsigned short fcw;
    __asm__ volatile (
      "fstcw %0; "
      "andw $-2, %0; "
      "fldcw %0; "
      : "=m"(fcw)
    );
    long double ld = 0;
    unsigned char data[10] = {
      0x5b, 0x01, 0x04, 0x5e, 0x85, 0x00, 0x00, 0x00, 0xd8, 0x59
    };
    __builtin_memcpy(&ld, data, 10);
    ld += 1;
    printf ("%Lg\n", ld);
    return 0;
}
lh_mouse@lhmouse-dev:~$ gcc test.c
lh_mouse@lhmouse-dev:~$ ./a.out
Floating point exception
lh_mouse@lhmouse-dev:~$ uname -a
Linux lhmouse-dev 4.9.0-0.bpo.2-amd64 #1 SMP Debian 4.9.18-1~bpo8+1 
(2017-04-10) x86_64 GNU/Linux
lh_mouse@lhmouse-dev:~$ gcc --version
gcc (Debian 4.9.2-10) 4.9.2
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
----------------------------------------

> Or perhaps it's just for older processors? AFAIK, the Intel x87 80-bit
> format is regarded as an extended precision format in IEEE 754-2008
> (Section 3.7), and trap representations are not allowed since all
> encodings must map to a floating-point datum (Section 3.2).
ISO C does not enforce compliance to IEEE 754 or IEC 60559 of `long 
double` (see WG14 N1570 F.2, note that Annex F is normative). An IEC 
60559 extended format is recommended, but not required. I don't think 
ISO C forbids existence of `trap representation` of `long double`.

> This encoding could either be interpreted like an unnormalized number
> (but then transparently normalized) as suggested by IEEE 754-1985, or
> it should encode a NaN.
x87 doesn't normalize it but leaves it invalid after loading it. `FXAM` 
doesn't say it is a NaN, so it is not a NaN.

-- 
Best regards,
LH_Mouse