Re: surprising precision

[Bob Plantz <plantz at cds1 dot net>]

Andrew Haley wrote:
> On 03/04/2010 03:53 PM, Steve Teale wrote:
> > On Wed, 2010-03-03 at 16:52 -0500, John S. Fine wrote: 
> > > Part of your confusion is over the difference between "precision" and 
> > > "accuracy".
> > >
> > > A floating point format has a FINITE number of specific values that can 
> > > be stored with infinite accuracy.  You happen to have chosen values that 
> > > are in that finite set.
> > >
> > > You can set the precision of your output to whatever (finite value) you 
> > > like.  It may be much more than the accuracy of your number.  It may be 
> > > much less than the accuracy of your number (especially if the accuracy 
> > > of your number is infinite).
> > >
> > > Even if the accuracy of a stored floating point number is infinite, it 
> > > may be possible for the algorithm that converts it from binary to 
> > > decimal to introduce some inaccuracy.  I'm not sure of those details.  
> > > Your results seem to indicate that translation is done surprisingly well.
> > >
> > > To help you understand, instead of outputting just d each time, output 
> > > both d and d+1.  At some point d will still be 100% accurate but d+1 
> > > will not be accurate, in fact it will be exactly equal to d.
> > John,
> >
> > Congratulations on a helpful answer. Yours was the first one that did
> > not display the almost conventional GCC guru arrogance.
> >
> > GCC would have a lot more participants/helpers if it were not for this
> > unwritten convention of rudeness in the community.
> >
> > To be fair, this isn't just a GCC thing. I've worked in various shops
> > where the software paid the wages, but if a new employee asked for help
> > he/she would get a similar response.
> >
> > I understand the impatience of the gurus. If they tried to answer every
> > naive question they'd never get any work done
> >
> > But, if you don't have anything helpful to say, then maybe it's better
> > to bite your tongue! The person you are responding to may be an idiot,
> > but he/she is not the only recipient of your message.
>
> I strongly disagree.  The main problem may have been that the OP
> didn't understand floating-point arithmetic, so the advice to find out
> about that was appropriate.  Let's see what the OP says.
>
> Andrew.

I agree with Andrew here, but I admit that at least my responses 
probably did not adequately answer the OP's question. My mistake is that 
I did not read his code carefully enough.

So I copied his code and tried it. I believe his real question is 
related to his not understanding that the function that displays a value 
is different from the storage of that value.

For example, let us say that I want you to write the number 555...5 
where there are N 5s in the number. Now if you and I agree that there 
are only 5s in this result, then I only have to tell you what N is. 
Using this protocol, I can specify a very, very large number with only 8 
bits. That is, I can specify a number up to 255 5s in only 8 bits.

Okay, floating point storage works a little like this in that a value is 
expressed as some fraction times 2 raised to a power, say P. In binary, 
2 raised to the Pth power is 1 followed by P zeroes.

OP, your program prints 2 raised to the 127th power. In binary this is 1 
followed by 127 zeroes. Now I can tell you then number of zeroes, up 
through 127, in only 7 bits. But when I convert 1 followed by 127 zeroes 
to decimal and write it out, that looks like it takes a LOT more than 7 
bits to express that number.

The key here is that using on these 7 bits, the next adjacent number is 
2 raised to the 126th power, a LONG distance from 2 raised to the 127th 
power.

So I think there are two things the OP should learn about:

1. How floating point numbers are stored.

2. That the conversion from a binary storage format to a decimal display 
on the screen is a separate function. Similarly, conversion from decimal 
input on the keyboard to an internal binary storage format is also a 
separate function.

In the spirit of full disclosure, I taught this material at the 
university level for over two decades and have written a textbook that 
covers both topics at the assembly language level. My textbook remains 
unpublished due to a perceived lack of market by commercial publishers, 
but it is still being used at the university where I taught. BTW, it 
uses the x86-64 architecture.

--Bob