Re: thoughts on martin's proposed patch for GCC and UTF-8

[Richard Stallman <rms at gnu dot org>]

    Yes.  Some widely used multibyte encodings use ordinary ASCII bytes to
    encode multibyte characters.  Examples include Shift-JIS, BIG5, and
    7-bit ISO-2022.

This makes the situation more difficult.

I think the key to how to cope with it is to recognize that currently
GCC does not handle these encodings at all.  But it does handle (in
strings and comments) multibyte encodings that use only non-ASCII
bytes, and it does so reliably, regardless of the environment.  I'll
call these "clean multibyte encodings".

The default mode in GCC should continue to handle clean multibyte
encodings reliably, meaning that it should not try to understand the
encodings, just treat sequences of non-ASCII bytes in the usual way.
Perhaps it will handle them thus in identifiers, as well as in strings
and comments.

However, it would be ok to add an option which tells GCC to decode all
multibyte encodings encountered, according to the specified locale.
That mode would handle the unclean multibyte encodings.

In addition to that, handling of \u in a non-wide string has to
depend on the encoding.  So there may be two things for which GCC
needs to know the encoding, and there is certainly at least one.

It is unreliable to get the encoding from the environment.  So we
should provide other ways to specify the encoding, and encourage
people to use them.  One way is with an option, --locale=LOCALE
Another way is with a directive such as

  #locale LOCALE

in the source code.

I think GCC should issue a warning if the source actually depends on
the choice of locale, and the locale has been obtained from the
environment.  The warning should encourage use of --locale or #locale
to specify the locale.

      I think \u will need
    to be translated, though, if possible -- unless the assembler handles
    \u, which is not true for gas at least.

Once we decide that either GCC or the assembler should translate \u
into a locale-specific multibyte encoding, it may as well be done in
GCC.  GCC is used with many different assemblers.

    * We'll have to disable the checking for identifier spellings in
      multibyte chars, since we won't know which multibyte chars are
      letters and/or digits.

Why would we want to check?  Why NOT simply define all non-ASCII
characters as being allowed in identifiers?  No non-ASCII characters
have any other meaning in C.

    * In general, assembly language files will not be text files.

When GCC wants to put certain non-ASCII bytes into a string or
identifier, that doesn't necessarily mean just outputting those bytes
into the .s file.

Non-ASCII bytes in strings can be output to the .s file using .byte,
so that the bytes themselves don't appear in the file.  This is a
reliable way to produce the same sequence of bytes in core when the
program runs.

Non-ASCII bytes in identifiers need to be encoded in some
way, since assemblers won't allow them in identifiers.

I suggest using `.' followed by the hex code of the byte.
That is allowed by most assemblers, and provides a unique
representation.  Of course, there should be a way to
specify a different handling for any given system,
in case the native compiler does something different.