Floating-point irregularities in ga68
Nelson H. F. Beebe
beebe@math.utah.edu
Mon May 18 14:33:19 GMT 2026
In a posting to this list on Sat, 9 May 2026 09:20:41 -0600,
I wrote about binary <--> decimal conversions:
>> ...
>> It is worth noting here that the required digit counts for correct
>> round-trip binary <--> decimal conversions have been known since 1968,
>> but most programmers are unaware of them, and almost all programming
>> languages get those counts wrong.
>> ...
A respondent questioned my "almost all" qualifier.
Here is a fragment from the book that I'm writing that gives evidence
for my statement:
>> ...
>> Regrettably, output conversions without precision specifications in
>> both Fortran and C default to 6 fraction digits for
>> floating-point output, whereas Matula mandates 9 for the 32-bit IEEE
>> format, 10 for older 36-bit formats with 27-bit fractions, and 17 for
>> the 64-bit IEEE format. Such defaults are likely to be relied on by
>> many users, who do not understand that subsequent input of those
>> truncated values will \emph{not} reproduce the numbers that were in
>> the computer, because they lose 3 or more trailing bits.
>>
>> More recent programming languages are little better. For the IEEE 754
>> 32-bit and 64-bit binary formats, without specifying an output
>> precision, COBOL produces 8 and 16 decimal digits, C\# outputs
>> 7 and 15, and Go, Java, and Rust output 8 and 17.
>> Some languages only provide the 64-bit format: the Awk, PHP,
>> and Python scripting languages all output 6 digits,
>> Algol 68 Genie, LibreOffice, and Perl output 15 digits,
>> Matlab and Octave output 5 and 16 in short and long formats,
>> GNU Pascal outputs 16 digits, and
>> JavaScript and MetaPost produce the Matula-mandated 17 digits.
>> Given the age of some programming languages,%
>> \footnote{Fortran appeared first in 1956, Algol 58 in 1958,
>> COBOL in 1959, Algol 60 in 1960,
>> Algol 68-R and Pascal in 1970, C in 1972,
>> and C++ in 1979.}
>> and given that several of them are defined by international standards
>> written by presumed experts in the languages and their practical
>> implementation, we have considerable evidence that correct round-trip
>> base conversion has been generally ignored, and the chosen defaults
>> are simply \emph{wrong}.
>> ...
In the manuscript, I comment further about the subtlety, and
difficulty, of the base conversion problem:
>> ...
>> When Donald Knuth designed and implemented the \TeX{} typesetting
>> system and the accompanying \MF{} font design system in 1977--1978,
>> for portability, he avoided floating-point arithmetic in favor of
>> fixed-point arithmetic in words of 32 or more bits. The low-order 16
>> bits supply the fraction part, and the high-order bit following the
>> sign bit is reserved for overflow detection, so there are 14
>> bits for the integer. He wrote code to convert between his internal
>> fixed-point binary format and the \emph{shortest possible} decimal
>> fractional numbers expected by users, and was confident that it was
>> correct, because with only 16 bits, all possible round-trip
>> conversions can be quickly tested. However, he was bothered for a
>> dozen years by a lack of formal proof, and finally was able to supply
>> one in a paper with the wonderful title \emph{A Simple Program
>> Whose Proof Isn't} \cite{Knuth:1990:SPW}. That encouraged another
>> prominent computer scientist, David Gries, to re-examine the problem,
>> and develop a different proof \cite{Gries:1990:BDO}. Both proofs show
>> that 5 decimal fraction digits suffice for 16 fraction bits;
>> Matula's general formula for arbitrary precision gives 6.
>> ...
The \cite{} command arguments refer to entries in this large, growing,
and actively maintained, BibTeX bibliography:
https://www.math.utah.edu/pub/tex/bib/fparith.bib
In a Web browser, you can change the final ".bib" in the URL to
".html" for a similar view, but with live hyperlinks. The file is
large --- more than 10MB, and more than 225,000 lines long --- so you
might view it better in your preferred text editor.
Search for the phrases "number base conversion" and "correct rounding"
to find many publications that have addressed the problem.
See also these additional entries for descriptions of the considerable
work needed to guarantee correct rounding:
Clinger:1990:HRF
Steele:1990:HPF
Burger:1996:PFP
Abbott:1999:ASS
Steele:2004:RHP
The Abbott entry observes on page 740 that the worst case for the
128-bit formats (about 34 decimal digits) requires 11_503 decimal
digits. That is why I suggested that ga68 would do well to follow gcc
and g++ in using the MPFR library for correct base conversions.
The IBM journal in which that paper was published was for decades
freely available, but a few years ago, its archives were moved behind
an IEEE paywall. I therefore put a pre-paywall copy for list readers
here:
https://www.math.utah.edu/~beebe/algol68/abbott-1999-ass.pdf
The others are in the ACM publication archive, which has been freely
accessible since last year.
----------------------------------------
P.S. If you find yourself doing lots of literature searching, I
recommend putting BibTeX data into an SQL database: see
BibTeX meets relational databases
https://tug.org/TUGboat/tb30-2/tb95beebe.pdf
That article has a tutorial on SQL searches, and the needed software
is highly portable and easily installed from
https://www.math.utah.edu/ftp/pub/bibsql
The SQLite3 database is by far the easiest to deal with, because it
requires no special privileges to use, and has over a trillion (yes,
10**12) installed instances. Alas, it does not support
regular-expression searching, which the SQL client/server versions do.
-------------------------------------------------------------------------------
- Nelson H. F. Beebe Tel: +1 801 581 5254 -
- University of Utah -
- Department of Mathematics, 110 LCB Internet e-mail: beebe@math.utah.edu -
- 155 S 1400 E RM 233 beebe@acm.org beebe@computer.org -
- Salt Lake City, UT 84112-0090, USA URL: https://www.math.utah.edu/~beebe -
-------------------------------------------------------------------------------
More information about the Algol68
mailing list