Re: GFortran BOZ-literals incompatible with F2003. Fix?

[Steve Kargl <sgk at troutmask dot apl dot washington dot edu>]

On Fri, Jan 12, 2007 at 01:36:30PM -0800, Brooks Moses wrote:
> Steve Kargl wrote:
> >On Fri, Jan 12, 2007 at 11:43:22AM -0800, Brooks Moses wrote:
> >>I just came across a spot where our GNU-specific extensions (as 
> >>documented in version 4.2 and up) for BOZ-literal constants are 
> >>incompatible with Fortran 2003, and I think we should reconsider this 
> >>before the 4.2 release.
> >
> >There is a PR about this.
> 
> Heh.  There's nothing new under the sun, clearly.  I'll look it up.

PR 18026

> [...]
> >>As I mentioned, the first version of GFortran that documents this 
> >>particular extension is 4.2, which we haven't released yet.  (I think it 
> >>may also be the first version that implements it, but I'm not sure.)  If 
> >>we do release it with this extension, we will then have a regression 
> >>against documented behavior once we implement this bit of F2003.
> >
> >The F95 behavior dates back to at least 4.1.0.  Documentation
> >has been a mess for a long time.  When we implement the F2003, we'll
> >need a -fF2003-boz option or some such, and we'll need to emit
> >a loud warning.
> 
> Yup; I'm wondering if it's better to have a -fF2003-boz option that 
> enables conformance to the F2003 standard, or to have a 
> -fboz-always-integer option that turns off the F2003-style conversions.
> 
> Personally, my feeling is that -std=f2003 should imply complete F2003 
> compliance (insofar as we've implemented it), and that -std=gnu should 
> be a superset of -std=f2003 that does not change the behavior of 
> standard-conforming programs.  Thus, I don't think the -fF2003-boz 
> option is the right way to go.

I can agree with the above.  This is partly why I thought a BT_BOZ
would be useful.  That is, we can have warnings for uses of BOZ
constants that are extensions as well as do what many people want
with setting Inf and NaN in data statements.

> I will note that I couldn't find any uses of REAL(Z'... in Google Code 
> Search at all, so either distinction is unlikely to affect real-world 
> existing code.  However, once F2003 becomes widespread, I suspect there 
> will be uses that it will affect.
> 
> >>My thought is that we should probably make this a legacy extension 
> >>rather than a GNU extension, or otherwise have some means of producing a 
> >>warning whenever a bare BOZ-literal is used.  Thoughts?  Differing 
> >>opinions?  Any ideas for how to implement both sanely without weird 
> >>special-casing?
> >
> >I looked at this briefly.  First, it needs to be mentioned that when
> >gfortran parses and recognizes a BOZ constant, it immediately converts
> >it to an integer.  So, by the time gfortran determines you have
> >REAL(...), the BOZ has already been converted.
> >
> >I tried to change gfortran have a BT_BOZ basic type (similar to hollerith).
> >Then, I added a "char *boz_str" to gfc_symbol.  My plan was to save the
> >BOZ string in match_boz_constant and delay conversion until needed.  I 
> >never got it work.
> 
> Yeah, that makes sense to me, too.  Although, since an integer is a 
> perfectly good representation for the boz-literal, I wonder if it might 
> not make more sense to make "BOZ_LITERAL" an integer kind rather than a 
> separate type.

I think that we should not mix a a boz kind into the integer model.
If I (or someone) can make BT_BOZ work, then in arith.c we'll have a
gfc_boz2int, gfc_boz2real, and gfc_boz2complex.  Each of these can
have various levels of warnings and errors.

> The question I meant to ask, though, was: How can we implement this on 
> the user's end of things in a sane way?  Do we simply allow both 
> extensions, but arrange things such that (when the F2003 extensions are 
> enabled, by whatever means) REAL(Z'FF') means one thing, and REAL(Z'FF' 
> + 0) means something entirely different?  Do we do this in -std=gnu, or 
> only in -std=legacy?

These are different.  REAL(Z'FF') involves a BOZ-literal-constant.
REAL(Z'FF' + 0) involves an expression and Fortran always evaluates
the (rhs) expression without knowledge of the lhs.  I would have no
problems with gfortran doing the Right Thing for REAL(Z'FF') with
respect to the F2003 standard, and issuing a warning for REAL(Z'FF'+0)
and doing the extension, ie.,

Warning: In file.f90:80:
    y = real(z'ff'+0)
             1
BOZ-literal-constant at (1) appears in nonstandard context. Undesired
behavior may result. 

> Also, are X = REAL(Z'FF') and X = Z'FF equivalent, or not?

This one is trickier.  One could again invoke that rhs expression
in x = z'ff' (even though it's a constant expression) should be
evaluated without knowing anything about the lhs.  This looks like
a never ending stream of bug reports.

> I don't see a good way to enable both that doesn't start getting really 
> twisted around the edge cases.

Unfortunately, I have to agree.  This could get rather nasty. :(

> >There is also some ambiguity with what to do with REAL(Z'FF').  Is
> >Z'FF = 0x0000000FF or 0xFF0000000?  What do we do with big vs little
> >endian?  I'll need to check again, but I believe we need to break
> >the BOZ in a significand and integer exponent to give the BOZ to
> >mpfr.  That is, mpfr expects a string of the form "0xACBp4" where
> >"p4" denotes 16**4.
> 
> Yeah, that's also a complication.  The standard says "processor 
> dependent", which is ... not helpful.  My thought is that we should 
> consider this to be parallel to the integer case (which is using the 
> bitwise model representation, effectively): pad the BOZ constant on the 
> left as needed, and use a model representation of the real numbers that 
> starts out at the right end with the exponent least-significant-bits to 
> most-significant bits, followed by the significand in least-significant 
> to most-significant.  This, at least, parallels all of the textbook 
> illustrations of floating-point numbers I've seen.

I haven't tried to locate the J3 rationale for the new F2003 
behavior, but I suspect that someone rammed this into the 
Standard because of shortcomings with TRANSFER.  It seems to be
an ill-concieved effort to deal with the Inf and NaN initialization
of real constants.

> Alternately, depending on what Paul does with his overlapping-array 
> stuff and TRANSFER, we could just effectively run things through 
> something like this:
> 
>     TRANSFER(INT(Z'FF', INT_KIND), REAL(0, RESULT_KIND))
> 
> Where INT_KIND has the same size as RESULT_KIND.  This may be tricky if 
> there is no INT_KIND large enough, though; one could then argue for not 
> doing the INT-to-bitwise conversion but just using the bitwise-to-REAL 
> part of the TRANSFER machinery.

I promised to help Paul with the transfer issue a long time ago.
Unfortunately, life has gotten in the way.  My idea was to use a
"unsigned int8_t *buf" to convert mpfr numbers into IEEE 754 
Uniform Binary Format (see a draft revision of IEEE 754).  The code
flow was

void
mpfr_to_binary (mpfr_t x, int kind, uint8_t **buf)
{
   char *str;
   mpfr_exp_t e;
   uint8_t *y;

   y = (uint8_t *) malloc(kind * sizeof(uint8_t)
   memset(y,0,..)
   sgn = get sign of x;
   
   if (x == inf)
     {
       y = set Inf bit pattern from IEEE 754
       if (sgn) y = set sign bit if need
       buf = &y
       return;
     }

   if (x == NaN)
     {
       y = set NaN bit pattern from IEEE 754
       if (sgn) y = set sign bit if need
       buf = &y
       return;
     }

   /* Get the binary string for significand and exponent. */
   mpfr_get_str (str, &e, ....)

   if (x < tiny)
    {
      subnormal number.  adjust str and e;
    }
   else
      Adjust e with a bias.

   Scan str to set bit in y
   Set exponent bits in y from e
   buf = &y;
   return;
}

Seems simply, right? :-)

-- 
Steve