RFC on matters related to straightening

[chris hermansen]

Good morning everyone,

On Mon, Jul 20, 2026 at 4:40 PM Jose E. Marchesi <jemarch@gnu.org> wrote:
>
>
> > Thanks for your speedy reply Jose,
> >
> > On Mon, Jul 20, 2026 at 1:51 PM Jose E. Marchesi <jemarch@gnu.org> wrote:
> >>
> >>
> >> Hi Chris.
> >>
> >> > Good morning everyone,
> >> >
> >> > Back from meddling with a new proc whole and before I'm ready to start
> >> > on a new proc fixed and proc float, I had some late-evening thoughts
> >> > about straightening.  This is a lengthy epistle, so if the thought of
> >> > reading over my wanderings through this topic seems less than
> >> > desirable, I apologize!
> >> >
> >> > There is a TL;DR down near the end of this message; please feel free
> >> > to skip there first.
> >> >
> >> > As far as I can tell, in the RR and in van Vliet's transput, the only
> >> > reason for inventing the concept of straightening is the desire to be
> >> > able to pass a value of mode OutType to print, write, put, putf, etc,
> >> > and similarly a variable of mode InType to read, get, getf etc. and
> >> > have the procedure able to deal with writing or reading said value or
> >> > variable without the need for the programmer to provide directions on
> >> > how to convert complex data structures to a row of simple values (int,
> >> > real, compl, bool, bits, char, or []char) or references thereto.
> >> >
> >> > In particular, the RR (and van Vliet follows the RR here), transput
> >> > operators STRAIGHTOUT and STRAIGHTIN "magically" convert values /
> >> > variables of mode OutType / InType to values / variables of mode
> >> > SimplOut and SimplIn, which are rows of simple types int, real, compl,
> >> > bool, bits, char or []char.
> >> >
> >> > Any disagreements so far?
> >>
> >> Sounds accurate to me.
> >
> > Good!
> >
> >>
> >> > Moving along, and picking up a couple of threads from past
> >> > conversations on this list, I see this business of STRAIGHTOUT and
> >> > STRAIGHTIN as a limited form of what O-O people call "object
> >> > introspection", that is, the ability to examine the mode of an object
> >> > at runtime in order to decide what to do with it.
> >>
> >> I personally wouldn't jump to associate straightening with
> >> introspection.  See below.
> >
> > Also good.  No claims that my interpretation is the correct (or only) one.
> >>
> >> > The conformity-clause is also a kind of introspection and we would use
> >> > it (for example) while handling the conversion of mode SimplOut to its
> >> > constituent bits, or the conversion of constituent bits into something
> >> > of mode SimplIn.  Something like:
> >> >
> >> > proc so_to_string = (SimplOut so) string:
> >> >    case so
> >> >    in (int x): whole(x,int_default_width),
> >> >       (real x):
> >> > float(x,float_default_width,float_default_after,float_default_exp),
> >> >       (compl x): float(re OF
> >> > x,float_default_width,float_default_after,float_default_exp)
> >> >                  + "+i" +
> >> >                  float(im OF
> >> > x,float_default_width,float_default_after,float_default_exp),
> >> >       (bool x): (x | "t" | "f"),
> >> >       (bits x): bits_to_string(x),
> >> >       (char x): string(x),
> >> >       ([] x): x
> >> >    esac;
> >> >
> >> > However, while the conformity-clause can work with rows, it doesn't
> >> > have the expressive power required to deal with arbitrary struct
> >> > values / variables.
> >> >
> >> > And so we need STRAIGHTOUT and STRAIGHTIN to "de-structure" struct
> >> > values / variables into rows.
> >> >
> >> > In sum, being able to code
> >> >
> >> > struct (int int_part, real real_part, bool bool_part) s;
> >> > int_part OF s := 1;
> >> > real_part OF s := 1.5;
> >> > bool_part OF s := false;
> >> > print((s, new line))
> >> >
> >> > requires:
> >> > - united modes
> >> > - conformity-clauses
> >> > - STRAIGHTOUT "magic"
> >> >
> >> > Still OK here?
> >>
> >> Yes.
> >
> > Awesome; for me at least, thinking of these as the three minimal and
> > sufficient things that must be properly implemented to be able to
> > proceed with proc print, putf etc is an important and fundamental
> > principle.
> >
> >>
> >> > Since the RR, the concept of "introspection" has been formalized and
> >> > generalized beyond what can be achieved with
> >> > union-conformity-STRAIGHTOUT/IN.  Today, we see introspection used not
> >> > only to handle this specific transput issue, but other similar matters
> >> > including:
> >> > - general serialization / deserialization:
> >> >    - converting a native data structure to/from JSON or XML,  or
> >> >    - converting a native data structure to/from an object-relational database;
> >> >    - such converters are often called CODECs (COnverter-DEConverter);
> >> > - runtime mode checking:
> >> >    - united modes and conformity clauses provide a partial runtime mode
> >> >       checking capability, but as noted previously, conformity-clauses don't
> >> >       address structure modes;
> >> > - useful runtime testing and debugging capabilities;
> >> > - the ability to do other things with arbitrary data structures with "generic"
> >> >    code (building libraries that implement lists, sets, hash tables etc of
> >> >    arbitrary structured data).
> >> >
> >> > One could argue that the last point above can be handled by
> >> > parameterized modes - and I would be sympathetic to that argument, as
> >> > I like parameterized modes, at least as implemented in Java generics.
> >> > But that's a bit off-topic right now.
> >> >
> >> > Could union - conformity - STRAIGHTOUT/IN handle converting a native
> >> > data structure to/from JSON or XML?
> >> >
> >> > As designed in the RR, I would say "no" - there's no way in
> >> > STRAIGHTOUT/IN to get the names of the fields in a structure.
> >> >
> >> > Obviously, operators STRAIGHTOUTWITHFIELDNAMES and
> >> > STRAIGHTINWITHFIELDNAMES could be provided that would help with that.
> >>
> >> You would need STRIGHTOUTWITHFIELDNAMESUNIONOVERHEADSANDBOUNDS.
> >>
> >> Straightening does not preserve _any_ of the structure of the original
> >> data , other than the order of the basic serialized constituents.
> >
> > Right, I get that.  An important point; the last step of converting
> > structures of structures is flattening the rows of rows (though that
> > may just be a pretty way to think about it; alternatively it could be
> > a plain old depth-first search of the structure).
> >>
> >> Thats precisely the point of it.  It is an anti-introspection device :)
> >
> > I'm ok with that interpretation; nevertheless, if rather than a
> > depth-first ordering of some arbitrary structure's values, what looked
> > more like JSON but flattened into a single row, we would be on our way
> > to introspection.  ¿No?
>
> Sorry, I don't understand that statement.

I'm thinking that, if we had something like STRAIGHTOUT/IN that,
instead of just providing a depth-first ordered row of the values in a
data structure, it provided a depth-first ordered row of tuples of
values, field names, and any other relevant metadata, we would have
something that looked very much like introspection.

I'm using "introspection" here as distinct from "reflection" - just
being able to read the useful metadata about a data structure along
with being able to read and write the values it contains.
>
> >>
> >> > On the other hand, I think that union - conformity - STRAIGHTOUT/IN
> >> > could be used to emulate at least some parameterized mode - related
> >> > tasks, by serializing / deserializing arbitrary data structures.  For
> >> > example, I could imagine
> >> >
> >> > proc hash_put = (ref HashTable ht, OutType key, value) void: {... uses
> >> > STRAIGHTOUT};
> >> > proc hash_get = (HashTable ht, OutType key, InType value) void: {...
> >> > uses STRAIGHTIN};
> >> >
> >> > The above creates the situation where a given HashTable can use keys,
> >> > and hold values, of any mode, which can be desirable, but also creates
> >> > run-time insecurity that could be eliminated with parameterized modes.
> >> >
> >> > Still, better than nothing!  And anyway, one can argue that
> >> > serializing and deserializing inherently creates run-time insecurity,
> >> > since it's difficult to ensure that changes to data structures be
> >> > maintained across applications that serialize and other applications
> >> > that deserialize "the same stuff".
> >> >
> >> > Still following?  Any thoughts?
> >> >
> >> > TL;DR
> >> >
> >> > Ok, so what I'm wondering is if the GNU Algol 68 provides:
> >> > - mode SimplOut;
> >> > - op STRAIGHTOUT;
> >> > - op STRAIGHTOUTWITHFIELDNAMES;
> >> > - mode SimplIn;
> >> > - op STRAIGHTIN;
> >> > - op STRAIGHTINWITHFIELDNAMES;
> >> >
> >> > that we could address not only transput as envisioned by the RR, but
> >> > other interesting and useful capabilities that we have learned about
> >> > since Algol 68 was designed to be run on a mainframe computer.
> >> >
> >> > What do you think?
> >>
> >> I see three general possibilities to implement straightening in ga68:
> >
> > This was what I wanted to hear!
>
> I really want to implement straightening so we can have transput :)

For sure!  My point is, besides "transput as envisioned by the
committee in the 60s and 70s", we can address other things that kind
of look like transput too - conversion to/from JSON, XML,
communication with databases, etc etc.
>
> >>
> >> a) Via explicit operators like you suggest,
> >> b) Via implicit coercions like in ALGOL68-RS,
> >> c) Via pragmat, i.e. compiler does the magic in such marked parameters.
> >>
> >> Somehow I am personally inclined to c).
> >
> > Could you give an example of what this c) might look like?
>
> Depends on how general we want to be.
>
> a) The simplest an less flexible way of doing this would be to have the
>    compiler to provide 'get' and 'put' as intrinsics, and doing the
>    straightening when it sees calls to get and put.  The routine texts
>    of 'get' and 'put' themselves would not be written in Algol 68 in
>    standard.a68.in, but emitted by the compiler instead.

I don't think this is a very good idea.  Yes it sounds simple at a
glance, but get and put are not simple pieces of code and they still
have conformity-clauses all the messing around with fields in File
objects and...

Plus there would then be the temptation (need?) to provide getf and
putf as intrinsics, which are even more complicated than get and put
and especially in relation to applying collections of Format objects
to the values to be printed or read.

In other words, moving a significant chunk of Algol 68 transput into
the compiler, and having to reflect any changes to File or other
transput-specific objects on the Algol 68 side into the compiler and
vice-versa... sounds like maintenance headaches to me.
>
> b) Something more flexibl would be to do something like:
>
>    mode SimplOut = union (char, bool, int, long int, ref char, ...);
>
>    proc put = (ref File f, []#[straight]# SimplOut x) void:
>       ...;
>
>    The compiler then:
>
>    - When compiling the call to such a procedure, it does the
>      straightening generating the multiple of simplouts.
>
>    - When compiling the routine text of such a procedure, it assumes a
>      []SimplOut is being passed.
>
>    This would allow to use straightening in other procedures, and `get'
>    and `put' woulnd't be special.
>
> I think b) would be better, although it requires more work to implement.

So your pragmat approach appears to be functionally similar to
Mountbatten's "STRAIGHT" used in the mode declarations for SIMPLIN and
SIMPLOUT, for example p.131 of PAME

MODE SIMPLIN=
   UNION(
    REF CHAR, REF[]CHAR,
    REF BOOL, REF[]BOOL,
    ...
    REF[]SHORT COMPL,
    STRAIGHT SIMPLIN
),

except of course hers was not a pragmat; nevertheless a device that
provides the ability to turn anything into SIMPLIN or SIMPLOUT in such
a way that we don't ever need modes INTYPE and OUTTYPE.

I like this kind of approach for a few reasons:

1. it deals with two modes - INTYPE and OUTTYPE - that are impossible
to implement in Algol 68 by simply eliminating the need for their
existence and using mode equivalence to do some of the heavy lifting
2. it honours the two modes - SIMPLIN and SIMPLOUT - as defined in the
RR in a way that's easy to fit into get and put as defined in the RR
or van Vliet, where instead of get  having parameter of mode INTYPE
and then converting with conformity-clause for INTYPE and []SIMPLIN y
= STRAIGHTIN it, the parameter is of mode SIMPLIN and the
conformity-clause uses SIMPLIN directly (and similarly for put /
OUTTYPE / SIMPLOUT, and for getf and putf)
3. more generally, it doesn't require the compiler to know about any
specific data structures like File; it just needs to know how to
straighten any data structure, which in the end sounds simpler and
easier to maintain than providing get, put, getf, putf as intrinsics
3. it suggests a path to follow for more "informative straightening"
where metadata is provided along with values of fields in arbitrary
data structures, assuming we ever want to do that.

>
> > I've reviewed Mountbatten's "BIOP 99" approach and there's something
> > about it that doesn't fire me up.
> >
> >>
> >> Which approach to take is a discussion that we need to have now, because
> >> otherwise you won't be able to finish the transput implementation..
> >
> > I think I can hardly even start it without some basic building blocks like this.



-- 
Chris Hermansen · clhermansen "at" gmail "dot" com

C'est ma façon de parler.