__builtin_isnanl() and invalid x87 80-bit floating point numbers

Liu Hao lh_mouse@126.com
Tue Apr 25 19:22:00 GMT 2017

Hello there,

mingw-w64 is having a potential bug here, but this seems related to GCC 
too. For your reference, details of the mingw-w64 problem can be found 
here <https://sourceforge.net/p/mingw-w64/bugs/602/>.

As for GCC, the following program:
int main(){
   long double ld = 0;
   unsigned char data[10] = {
     0x5b, 0x01, 0x04, 0x5e, 0x85, 0x00, 0x00, 0x00, 0xd8, 0x59
   __builtin_memcpy(&ld, data, 10);
   __builtin_printf("nan? %d\n", __builtin_isnanl(ld));
outputs `nan? 1`. This has been confirmed to be the case on GCC 6.3 on 
x64 Windows and GCC 4.7 & 4.9 on x64 Linux.

According to 
the number in question is an invalid operand (effectively a _trap 
representation_), because its exponent is neither all-zeroes nor 
all-ones and its bit 63 is zero. Unlike NaNs, this invalid number can't 
be copied as a `long double`. x87 loads it successfully using the `FLD` 
instruction, but generates an #IA (invalid operand) exception if `FST` 
is used to store it into RAM, which isn't the case for a SNaN or QNaN.

If the `FXAM` instruction is used to classify this invalid number, we 
get `[C3:C2:C0] = 0'b000` in the x87 status word, saying the number is 
unsupported, while we would get `[C3:C2:C0] = 0'b001` if it were a NaN.

I have also checked the assembly output of `__builtin_isnanl()`. It 
seems that GCC uses the `FUCOMP` instruction to compare it with itself. 
Because on Windows the #IA exception is masked by default, no exception 
is visible to the program, and we get the result 'unordered', and GCC 
thinks being 'unordered' means being a NaN, which is doubtful.

Best regards,

More information about the Gcc-help mailing list