Byteswapping floating point types

[Andrew Troschinetz]

On Jul 15, 2009, at 3:44 AM, Andrew Haley wrote:

> On 07/14/2009 10:44 PM, Andrew Troschinetz wrote:
>> as opposed to a methods of byteswaping by leveraging unions
>> which breaks aliasing rules.
>
> Doing this with unions doesn't break aliasing rules on gcc.

Right. And I believe this is the case on most (all?) major compilers  
as well. I was just pointing out that the actual act of byteswapping  
isn't my concern, but rather the signature of the byteswap() function  
template which implies byteswapped floating point values are returned  
as floating point types.

>> But for floating point types doesn't this signature run the risk of  
>> the
>> compiler putting a byteswapped floating point value into an FPU
>> register, thus causing the data to be modified?
>
> Why would it?  Perhaps you're concerned that loading into a FPU
> register would convert a signaling NaN into a quiet NaN, or
> somesuch?

It is not an area in which I am well versed so I probably can't give  
you as good an answer as others on this list, but I'll give it a go.  
"Why would it?" isn't actually a valid question, you should be asking  
"why does it?" because all the tests I've done show that it does.

I believe the problem is that not all values of the sign, exponent,  
and mantissa of a floating point number are valid under IEEE 754. When  
you byteswap a valid floating point value you are very likely to get  
an invalid IEEE 754 floating point value, especially with single  
precision types.

When such an invalid floating point is loaded into a FPU register,  
bytes in the mantissa will be flipped in order to make the value NaN.  
You are correct that one of the dangers is that we could get a  
signaling NaN, though I don't know how likely that is.

However I have only seen this problem on when I attempt some  
mathematical operation on a byteswapped floating point, which is  
clearly something no one should ever be doing in the first place. My  
concern is that it the byteswap() function template isn't _safe_ on  
GCC or any other compiler because the compiler is free to put a  
floating point value into a FPU register as it sees fit. And we  
shouldn't rely on it not doing so.

>> My concern is that the compiler is free to put floating point values
>> into FPU registers as it sees fit and relying on the compiler not  
>> doing
>> so is dangerous.
>
> You're in the realm of at best implementation-defined or at worst
> undefined behaviour.  The CPUs I know are quite happy to do exact
> copies of memory via floating-point registers, and some versions of
> memcpy() make use of that.  But in general it's not guaranteed.

Interesting, though I saw similar problems with Ian's implementation  
which uses memcpy(). Our target platforms are (at the moment) x86_64  
and i686.

--
Andrew Troschinetz
Applied Research Laboratories