RFC positions in streams of text

[chris hermansen]

Good morning,

On Wed, Jul 29, 2026 at 2:27 AM Jose E. Marchesi <jemarch@gnu.org> wrote:
>
>
> > Huh! Interesting questions below...
> >
> > On Tue, Jul 28, 2026 at 5:12 PM Jose E. Marchesi <jemarch@gnu.org> wrote:
> >>
> >>
> >> Hi Chris.
> >>
> >> > Good afternoon everyone,
> >> >
> >> > I'm more or less back in the thick of things with transput.  One of the core
> >> > concepts in both the RR and van Vliet is that of "position" in the text of the
> >> > file, which is defined in the RR on p.141 as:
> >> >
> >> > {private} mode Pos = struct (int p, l, c);
> >> >
> >> > This triple is meant to indicate:
> >> > - the page number within the text;
> >> > - the line number within the current page;
> >> > - the character number within the current line.
> >> >
> >> > Back on 20 December, Jose and I had an exchange about transput where he
> >> > offered some opinions about the above.  He stated:
> >> >
> >> >> For character transput, structuring the files being edited in
> >> >> pages/lines/char is something that currently every application wishing
> >> >> to edit text structured data have to do by themseves, on top of the
> >> >> simplistic stream-of-bytes I/O systems: you should handle newline, you
> >> >> should keep count of the number of lines you are reading or writing,
> >> >> which column I am right now, etc.
> >> >>
> >> >> IMO it is cool for the transput system to implement the complexity of
> >> >> dealing with structured text.  Like so many other things, I/O systems
> >> >> got simplified way too much.
> >> >>
> >> >> One may argue about pages.  With some few exceptions (like the GNU
> >> >> custom of using ^L characters to group related information) almost
> >> >> nobody bothers to structure their text files into pages.  So we may want
> >> >> to simplify the book to be a collection of lines/chars, rather than
> >> >> pages/lines/chars.
> >> >>
> >> >> However, nowadays paging made a comeback by being used extensively in
> >> >> contents provided by web servers, where the result of a query is very
> >> >> often paged, to be fetched page by page.  So we may want to keep the
> >> >> page/line/char after all.
> >> >
> >> > I'm coming to the conclusion, after thinking about this on and off for quite
> >> > awhile, that I should either
> >> > - follow Mountbatten (dispense with page and line numbers except for channels
> >> >   like tapes or screens where that could make sense) or Marcel (just generally
> >> >   dispense with page and line numbers completely); or
> >> > - emprace them completely.
> >> >
> >> > Thinking about "embracing them completely" for a minute - to me this means
> >> > that in files that are streams of text (ie not binary mode)
> >> >   - use "'f" (form feed) as the way to separate pages
> >> >   - use "'n" (line feed) as the way to separate lines
> >> >
> >> > To do this when writing or reading text files sequentially, I need to count
> >> > form feeds and line feeds, call line_ended when reading or writing a line
> >> > feed and call page_ended when reading or writing a form feed.
> >> >
> >> > When reading sequentially, the set procedure will require reading character by
> >> > character, counting line and form feeds until the (p, l, c) values passed to
> >> > set are reached.
> >>
> >> The sequential nature of the operation is also required because as you
> >> observe below the stored data is UTF-8.
> >>
> >> > When writing sequentially, this type of "set by reading" is not feasible on
> >> > stand_out_channel, since it may not be opened for reading + writing.  It may
> >> > be feasible to work this way when writing sequentially on stand_back_channel,
> >> > but only when the desired set position is before the current write position
> >> > (which is the end of file by definition if I am correct).
> >>
> >> This is interesting.
> >>
> >> According to the RR the requirements for a File opened through
> >> stand_out_channel ("a channel value whoe field selected by 'put' is a
> >> routine which always returns true, and whose other fiels are some
> >> suitable values") is that put_possible should always be true.
> >
> > I've read this over a few times in both the RR and van Vliet and I guess
> > really there's no reason to "forbid" stand_out_channel from opening files
> > for both reading and writing, at least when connected to a filesystem file.
> >
> > I must have been reasoning along the line that "stand_back_channel must
> > support both writing and reading, whereas stand_out_channel must support
> > writing, ergo stand_out_channel does not support reading".  Bah and humbug.
> >
> > Having said that, we cannot arrange things so that put_possible is always
> > true on stand_out_channel, because a filesystem file may not be writeable.
> >
> > So the RR specification of the behaviour of stand_out_channel cannot
> > be implemented.
>
> Well, in that case the open fails, no?

The open fails, returning a non-zero integer that is intended to explain
why.  Van Vliet vol.2 p. 62-64 explains "If opening is not successful,
a non-zero error code is returned" and provides a list of error codes
that makes a good basis for moving forward.

However, transput calls must guard against an unopened file (whether
the call to open was not issued, or was issued but failed).

>
> >
> >>
> >> a) The filesystem file can be read.  In this case set_possible for the
> >>    File is true.
> >
> > That seems completely reasonable to me.
> >
> >>
> >> b) The filesystem file cannot be read (it is write-only).  In this case
> >>    set_possible for the File is false.
> >
> > That also seems reasonable.
> >>
> >> In other words: set_possible is contingent to the underlying filesystem
> >> file to be readable.  I think this would be conformant to the RR.
> >>
> >> WDYT?
> >
> > Seems ok.  A few points:
> >
> > First, Mountbatten PAME 13.7.2 p. 255 has:
> >
> >> The stand_out_channel is the standard buffered output channel. Books on
> >> this channel have the following properties:-
> >>
> >> bin_possible   TRUE
> >> put_possible   TRUE
> >> get_possible   FALSE
> >> set_possible   TRUE
> >> reidf_possible FALSE
> >
> > But then again, she ignores page and line counts on filesystem files,
> > which allows her to use seek() to move to a given character position.
>
> Well, it is more that in ctrans `char' are simply bytes.  The reason why
> we need read access for setting the position is that the stored UTF-8
> requires reading from the beginning in order to count characters.

When we think about when we might want to reposition the reading or
writing point, in our program we would know we want to (for example)
back up to a marked position.  But what the program thinks of as the
(Unicode) position (p, l, c) bears no relation to where the (UTF-8)
character is located in the filesystem file's byte stream.

So if we are to provide direct access, do we need a (hidden?) function
that maps Unicode position to byte stream location of the UTF-8 character?

If we are going to handle repositions in version 1 by going back to
the beginning and reading until the (p, l, c) position is equal to the
desired value, we don't have to worry about this, but I think we should
at least remember whatever we conclude here...

>
> By the way, we can make the above more efficient if we cache and manage
> the byte-locations of the beginning of lines in the Book, and other
> trickery :)

True, which relates to the mapping function I mentioned above.

>
> >
> > Second, if I write
> >
> >     File f;
> >     open(f, "/etc/passwd", stand_out_channel)
> >
> > open will fail, and subsequent calls to
> >
> >     put_possible(f)
> >
> > will return false.
>
> Yes, I would expect that.
>
> Isn't there another predicate that tells you the file is not opened?
> i.e. after an "open", how do you check the operation succeeded?

van Vliet vol.2 p.29 introduces "file status", which includes whether
or not the file has been opened. He then goes on to state:

> The present model supposes that it is always possible to determine
> whether a given file has been opened . So, after a declaration
>     FILE f,
> or even
>     REF FILE f = SKIP,
> it must be possible to detect that 'f' is not opened.  To this end, all
> user-callable routines may be assumed to start with an implicit test for
> the 'cover' field [of the File] being available. It is not defined here
> how this can be done in an actual implementation.

Given the above, I think the only ways to know are:
- look at the return code of open, establish or create; or
- try a transput operation to see if it works.

>
> >
> > Similarly, the RR definition of mode Channel includes the field
> >
> >     proc (ref Book) bool put
> >
> > which, contrary to the definition of stand_out_channel, would not return
> > true if given the Book corresponding to "/etc/passwd" resulting from the
> > call to open above.
>
> Aren't these ("set", "put", etc) just the default values to be set in
> the File's possibles _in the case the open succeeds_?

Could be.

>
> >
> > Third, van Vliet has a slightly different definition of stand_out_channel:
> >
> > Channel stand out channel =
> >   { a channel value such that for each file successfully opened on this
> >   channel, 'put possible' and 'backspace possible' always return true
> >   (unless the book linked with the file explicitly indicates otherwise),
> >   while the other environment enquiries for files return some suitable
> >   values };
> >
> > This is more useful in that it admits to the possibility of unwriteable
> > files, but also introduces the need to support backspacing.  I would
> > claim that backspacing won't be possible if the filesystem file is
> > not readable.
>
> This "unless the book linked with the file explicitly indicates
> otherwise" is interesting.  Does Van Vliet explains this in the Vol 1 of
> his thesis?

I don't see anything about that in vol.1.
>
> >
> >>
> >> > As for reading and writing at random locations throughout a file, I need to
> >> > think about this a bit more.  I guess by definition write at a random
> >> > location does not move the end of file.
> >> >
> >> > And as for binary files, I'm not addressing them as yet.
> >> >
> >> > So in sum, I guess I'm leaning into the "embrace completely" camp, at
> >> > least as far as dealing with text in filesystem files.
> >>
> >> I agree it is better to embrace the 3D view of the textual space of
> >> transput files.
> >
> > Good!  I think :-0
> >>
> >> > NB what I am counting are Unicode characters in the program and not
> >> > UTF-8 characters in the filesystem (which also seems kinda weird).
> >> >
> >> > Any comments, doubts?
> >>
> >> What about allowing the user to configure the line terminator and page
> >> terminator characters, with suitable Unicode defaults like \n and \f?
> >
> > No reason not to provide this capability that I can think of.
> >
> > Though best to review make_term before I go out on a limb.
> >
> > Thanks a lot; sorry I had to bail early from the meetup.



-- 
Chris Hermansen · clhermansen "at" gmail "dot" com

C'est ma façon de parler.