Re: thoughts on martin's proposed patch for GCC and UTF-8

[Paul Eggert <eggert at twinsun dot com>]

   Date: Thu, 17 Dec 1998 19:10:24 -0700 (MST)
   From: Richard Stallman <rms@gnu.org>

   One way is with an option, --locale=LOCALE
   Another way is with a directive such as `#locale LOCALE' in the source code.

Like your other suggestions, this one sounds reasonable to me.  It
might help to use more specific names, e.g. `--locale-ctype=LOCALE'
and `#locale LC_CTYPE LOCALE'.  Locales are a more general notion than
just their LC_CTYPE component, and we may find a use for specifying
other (non-LC_CTYPE) parts of the locale later.

   Why NOT simply define all non-ASCII characters as being allowed in
   identifiers?  No non-ASCII characters have any other meaning in C.

OK, you can talk me into not checking, at least by default.  It might
be useful to have an optional check, for people who want to port their
code to more restrictive compilers that check the restrictions of
draft C9x's Annex I.  This optional check would be locale-dependent.

   Non-ASCII bytes in identifiers need to be encoded in some
   way, since assemblers won't allow them in identifiers.
   I suggest using `.' followed by the hex code of the byte.

It may make sense to use this notation (or something like it) even
for assemblers like GAS that allow multibyte chars in identifiers.  This
would mean that assembly language files would be text, which is good.

However, a drawback of this approach is that the names will be mangled.
This will require that debuggers demangle the names.  For C++ this is
not such a big deal, as the names are already mangled, but for C it
is a bit inconvenient.

Here is another possibility.  For identifier chars that can be
expressed as multibyte chars in the locale's encoding, use those
chars; otherwise, use `.uxxxx' or `.Uxxxxxxxx' where xxxx (or
xxxxxxxx) are the Unicode position.  E.g. if the original identifier
was a\u1234b\U12345678c, and if \u1234 and \U12345678 cannot be
represented as multibyte chars, then represent this identifier as
a.u1234b.U12345678c in the assembler file.

An advantage of this approach is that (for C, at least), it's upward
compatible with martin's proposal.  A locale that uses a UTF-8 charset
and encoding will simply use UTF-8 identifiers, which is what he
wants.  This will be a natural way to do things in the UTF-8 world --
the assembler file will be much easier to read as UTF-8 text than it
would be with .uxxxx or .xx.xx.xx escapes.  (Similarly for other
encodings like EUC-JIS.)

I don't know how this would affect C++ mangling, though.

   handling of \u in a non-wide string has to depend on the encoding.

A minor point: this is also true for \u in wide strings, as not all
systems use Unicode for wide chars.