Variable assignment and truncation of floats on x86

[Geoff Keating]

Hi Joe,

I looked at your program.

The problem you are having is that 7/20 in floating point is
equivalent to (all numbers here starting with 0x are in base 16)
0x0.59999999999....  and so cannot be represented exactly in a finite
number of binary digits.  'float' has 24 significant bits, counting
the leading 1 bit, and so this value is rounded to 0x0.5999998, which
is slightly less than the true value.  When this is multiplied by 20,
the true result is 0x6.FFFFFE exactly, which you'll notice is still
less than 7.  When this value is rounded to 24 significant bits, it
becomes 7 exactly.

Now, in your program, you have:

  imm = ftmp = fll * (float)icc;
  immO = ftmp;

which is equivalent to

  imm = fll * (float)icc;
  ftmp = fll * (float)icc;
  immO = ftmp;

that is, the assignment to 'imm' is direct, but the assignment to
'immO' explicitly stores the result out to a variable of type 'float'.
ISO C permits an implementation to evaluate a direct assignment (of
the kind made to 'imm') in more precision than the operation would
imply, and ISO C99 provides the FLT_EVAL_METHOD flag to say whether
this will be done.  On x86, FLT_EVAL_METHOD is 2 which means that all
operations are computed in the full 80-bit precision allowed by the
FPU if no explicit rounding is specified.  However, GCC actually
doesn't get this quite right, and will often use the 80-bit precision
even when there are explicit casts to 'float' or intermediate
variables are used, and will sometimes round when according to
FLT_EVAL_METHOD it shouldn't.  This is a known deficiency of GCC on
the x86 platform.  There are other problems inherent in the design of
the x86 FPU that appear if you use variables of type 'double'.

Finally, when you cast a floating-point value to an integer value, it
is truncated towards zero.  Thus, 0x6.FFFFFE becomes 6.

Now, it is not clear from your message whether you expected the
rounding behaviour to occur as it did.  If you are really relying on
proper IEEE rounding for some reason on x86, my best advice is that
GCC is probably not the compiler for you (or x86 is not the
architecture for you) in its present form.  Of course, the GCC group
would very much welcome someone to come in and try to fix this, it's
kind of annoying that every other architecture can do proper IEEE
arithmetic but not x86.  A fourth choice is that you can adapt your
program allowing for slightly greater rounding error than IEEE754
would provide.  More likely, though, you were unaware that this
rounding was going on, in which case you probably need to learn more
about how floating-point arithmetic really works or one day you'll get
a really nasty surprise.

One last bit of advice: if you want a way to round a floating-point
value to the nearest integer, the function you want is called 'rint'.

-- 
- Geoffrey Keating <geoffk@geoffk.org>