Floating point performance issue

David Brown david.brown@hesbynett.no
Tue Dec 20 22:32:00 GMT 2011


As someone else has pointed out, this is an age-old argument that will 
never be settled.  I have no hopes of changing anyone's opinion here, 
but perhaps the exchange of opinions will be informative.

On 20/12/11 17:14, Vincent Lefevre wrote:
> On 2011-12-20 16:04:05 +0100, David Brown wrote:
>> But I think compiler flags are a suitable choice as they make it
>> easier for the user to apply them to the program as a whole.
>
> This can be done with pragmas too.
>
>> An example of this would
>> be "-fshort-double" or "-fsingle-precision-constant".  These specifically
>> tell the compiler to generate non-conforming code, but are very useful for
>> targets that have floating-point hardware for single-precision but not for
>> double-precision.
>
> If the developer wants to allow the user to choose the precision
> (or the type), he can do something like:
>
> #ifndef FP_TYPE
> /* default */
> # define FP_TYPE double
> #endif
> typedef FP_TYPE fp_t;
>
> or something more restrictive. At least, it is clear that the choice
> of the precision is allowed in the program.
>

I understand what you are saying here - and I agree that it's very 
important that any such choice is made clear and explicit.  But that's 
why a program's makefile is part of its source code - compiler flags are 
an essential part of the source.  Sometimes making use of specific types 
like this is a better way, but not necessarily - it may make the code 
less clear, it leaves more chance of accidentally mixing in doubles 
(such as from a constant without the "f" suffix), and it means you can't 
mix in pre-written code that uses "double" without changing it.  The 
"-fshort-double" flag is a "use once and forget" way to get much faster 
code (on relevant cpus, of course) at the cost of lost precision.


>>>> I would still say that most floating point code does not need such
>>>> control, and that situations where it matters are rather
>>>> specialised.
>>>
>>> I think that if this were true, there would have never been an
>>> IEEE 754 standard.
>>>
>>
>> The IEEE 754 standard is like many other such standards - it is very useful,
>> perhaps critically so, to some users.  For others, it's just a pain.
>>
>> I work with embedded systems. [...]
>
> OK, so I would say that embedded systems are very specific and do not
> concern most floating-point code. However it's a pity that people from
> the embedded world didn't take part of the discussion of IEEE 754R.
> IIRC the only discussions about embedded systems concerned FTZ, and
> IEEE 754-2008 has something similar (substitute / abruptUnderflow).
>

The embedded world doesn't tend to have good representation on these 
sorts of things - I agree it's a pity.

And while there are relatively few embedded programmers compared to "big 
system" programmers, it's still very relevant to gcc.  Most gcc users 
may be x86/amd64 target users, but the majority of gcc targets are for 
embedded systems, and they are the part of what drives gcc onwards.

>> My believe is that for the great majority of users and uses, floating point
>> is used as rough numbers.
>
> But the IEEE 754 standard is also a benefit for them (reproducibility
> of the results, portability, detection of potential problems thanks
> to exceptions...).
>

The programs would be equally portable if the last few bits of 
calculations varied, or the rounding was different on different 
machines.  And people do not expect bit-perfect repeatability of 
floating point, especially not across targets.

In my opinion, code that relies on exceptions to spot errors in 
calculations is normally bad code.  You don't do a division and handle 
divide-by-zero errors - you write code so that you never try to divide 
by zero.  At best, such exceptions are an aid during debugging.

>> People use them knowing they are inaccurate, and almost never
>> knowing or caring about exactly how accurate or inaccurate they are.
>> They use them for normal, finite numbers. Such users mostly do not
>> know or care what IEEE 754 is.
>
> Still, -ffast-math can be inappropriate for them due to enabled
> options like -fassociative-math. Exception handling can allow them
> to detect bugs or other problems.
>

Again, -fassociative-math is not a problem for code that does sensible 
calculations.  It's theoretical only, or for people who want to insist 
on bit-perfect repeatability of their floating point code.  The example 
case given in the gcc manual is "(x + 2**52) - 2**52)".  Assuming the 
implication is that x is a small number, there are almost no real-world 
circumstances when such code exists.  And if you really want to add or 
subtract numbers that differ by 52 orders of binary magnitude, and are 
interested in accurate results, you don't use "float" or "double" types 
anyway.

And again about exceptions - I work in a world where bugs cause big 
problems.  It would not be acceptable to have an attitude that you do 
the calculations regardless of the input, and let exception handlers (or 
signal handlers) clear up the mess once it all flops.  The rule is 
garbage in, garbage out - if you don't want garbage out, don't put 
garbage in.  Then the guts of your code can be written to assume that 
the input is valid, and the output will be valid.  While embedded 
programmers are often more vigilant than desktop programmers, I fail to 
see why any other sort of programmer would accept floating point 
exceptions as appropriate behaviour for code.

>> That's probably one of the most common mistakes with floating point
>> - the belief that two floating point numbers can be equal just
>> because mathematically they should be.
>
> No, the problem is more than that, it's also about the consistency.
> Assuming that
>
> int foo (void)
> {
>    double x = 1.0/3.0;
>    return x == 1.0/3.0;
> }
>
> returns true should not be regarded as a mistake (even though it might
> be allowed to return false for some reasons, the default should be
> true).
>

You should not be able to rely on code like that giving either true or 
false consistently.  You say yourself it "should return true" but "might 
return false".  The code is therefore useless, and it would be a good 
idea for the compiler to warn about it.

>> This "bug" is not, IMHO, a bug - it's a misunderstanding of floating
>> point.
>
> Despite some users who may misunderstand FP, it's really a bug that
> affects correct code. And the fact that some users do not understand
> FP is not a reason to make their life difficult.
>
>> Making "-Wfloat-equal" a default flag would eliminate many of these
>> mistakes.
>
> No. This bug is due to the extended precision. Floating-point code
> affected by rounding errors vs discontinuous functions (such as ==)
> will have problems, whether extended precision is used or not.
>

Yes, I understand the nature of the bug mentioned - floating point 
hardware may use more than "double" precision during calculations.  But 
I don't see why it should be treated any differently from any other 
mistaken attempts at comparing floating point numbers.



More information about the Gcc-help mailing list