[Bug c/12955] New: Incorrect rounding of soft float denorm mul/div

["eager at mvista dot com" <gcc-bugzilla at gcc dot gnu dot org>]

The software floating point emulation in fp-bit.c incorrectly rounds the results
of certain multiplications and divisions, especially those which involve
denormalized FP values.  The incorrect results are wrong in the low-order bit.

Rounding in fp-bit.c is done in pack_d(), which scales the value, discarding
guard bits and rounding as needed.  The root cause of the incorrect rounding is
that _fpmul_parts() and _fpdiv_parts() calculate an "infinite precision" result
which has more guard bits than can be passed to pack_d().  They attempt to
anticipate how the rounding will be performed in pack_d() and adjust the value
to account for discarded guard bits, but under some circumstances get this
wrong.

The error occurs when the guard bits which are passed to pack_d() are all zero
but there are additional non-zero guard bits which have been discarded.  Under
some circumstances, _fpmul_parts and _fpdiv_parts miscomputes the rounding and
incorrectly increments the bit which will be the low-order bit of the result. 
In other cases, it just passes all zero guard bits to pack_d(), ignoring the
non-zero discarded bits.  pack_d() then rounds incorrectly, assuming that the
value really does have all guard bits zero.

This isn't the right thing to do, anyway.  The multiply and divide routines
should only do that, not attempt to duplicate or second guess the rounding
decisions in pack_d().  The question becomes, what should be done with the
discarded guard bits.  IEEE-754 says that they cannot be ignored.

The solution is to have the low-order bit of the guard bits which are passed to
pack_d() represent all of the discarded guard bits.  If any of these discarded
bits are one, the low-order guard bit is forced to one.  This only makes a
difference if all of the higher order guard bits are zero.  If all guard bits
are zero (including the low-order and all discarded guard bits) then the result
is supposed to be rounded so that the low-order bit of the result is zero.  If
the guard bits are non-zero, then the value is rounded up or down as required by
IEEE-754.  

The attached patch does just this.  It has been tested by (1) passing the test
cases, (2) comparing single FP results with hardware generated results on x86,
and (3) comparing both single and double FP results on PPC with hardware FP.

-- 
           Summary: Incorrect rounding of soft float denorm mul/div
           Product: gcc
           Version: 3.3.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: c
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: eager at mvista dot com
                CC: gcc-bugs at gcc dot gnu dot org


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12955