Question - internal representation of char in Algol 68

[Bruno Haible]

Jose E. Marchesi wrote:
> > There appears to be some difference between UTF-32 and UCS-4; see this
> > article
> >
> > https://stackoverflow.com/questions/30186631/what-is-the-difference-between-utf-32-and-ucs-4
> >
> > in which comments indicate:
> >
> > UCS-4 and UTF-32 are now identical except that the UTF-32 standard has
> > additional Unicode semantics

This wording is nonsense.

UCS-4 and UTF-32 are now identical string encodings. Period.

No one is supposed to make a distinction between "a Unicode character with
Unicode semantics" and "a Unicode character without Unicode semantics". That
would just be nonsense.

> For `char' We are effectively using an unsigned 32-bit value, and then
> using code from GNU libunistring that interprets that value as:
> 
>  -- Type: ucs4_t
>      This type represents a single Unicode character, outside of an
>      UTF-32 string.
> 
> 
> Knowing the author of libunistring, who I am adding in CC, I am sure
> that formulation is very precise and not casual.

The intent of this formulation is to make a distinction between a
32-bit value in a variable (i.e. outside of a string — no endianness or
alignment issues) and an element of an UTF-32 string (which has endianness
and alignment, because it's part of memory).

> Seems to me that an UCS4 becomes UTF-32 only in relation to other
> "neightbourghing" characters..

Yes, that's a way to view it.

> > Not knowing the depth of collation rules, I can imagine there would be
> > circumstances where we might want to apply collation rules for comparing
> > two strings and other circumstances where we don't.  I further suppose that
> > would require two different operators, maybe EQ and COLLATESTO?
> 
> We have no choice hen it comes to string equality with =.  The semantics
> are based on comparing the ABS values of each 'char' ...

Right. Collation is tied to a locale, e.g. collation of strings in French
and in Spanish is different [1]. But a string equality operator like '='
has only 2 operands and no context. Therefore, yes, we have no choice.

Bruno

[1] https://en.wikipedia.org/wiki/Alphabetical_order#Language-specific_conventions