[RFC] Use wide chars to represent Fortran source internally

[Tobias Burnus]

FX wrote:
> This patch is a first step to handling non ASCII encoded source files 
> and non-default character kinds in gfortran.
>   -- in the routine handling backslash-escaped sequences when the 
> -fbackslash option is used, also handle \x??, \u???? and \U???????? 
> wide-char escape pattern
Can you update the man page? It currently reads:

-fbackslash
   Change the interpretation of backslashes in string literals from a 
single backslash character to "C-style" escape characters.  The 
following combinations are expanded \a, \b, \f, \n, \r, \t, \v, \, and 
\0 to the ASCII characters alert, backspace, form feed, newline, 
carriage return, horizontal tab, vertical tab, backslash, and NUL, 
respectively.  All other combinations of a character preceded by \ are 
unexpanded.


For instance as follows (but there are surely nicer versions):

Index: invoke.texi
===================================================================
--- invoke.texi (Revision 134991)
+++ invoke.texi (Arbeitskopie)
@@ -242,8 +242,10 @@
 The following combinations are expanded \a, \b, \f, \n, \r, \t,
 \v, \\, and \0 to the ASCII characters alert, backspace, form feed,
 newline, carriage return, horizontal tab, vertical tab, backslash,
-and NUL, respectively.  All other combinations of a character preceded
-by \ are unexpanded.
+and NUL, respectively. Additionally, @code{\x}@var{nn}, @code{\u}@var{nnnn}
+and @code{\U}@var{nnnnnnnn} (where @var{n} is a hexdecimal digit)
+are translated into the corresponding character. All other combinations
+of a character preceded by \ are unexpanded.

 @item -fmodule-private
 @opindex @code{fmodule-private}


>   -- modify show_locus to display large character source lines, by 
> escape-encoding characters that are not ASCII-printable (extending 
> what was previously done)
I think we need to update the algorithm. As soon as non ASCII characters 
are used, the error location is wrong.

Example:

print *, 'Hello World'wrong  ! position is OK
                     1
Error: Syntax error in PRINT statement at (1)

print *, 'Hello W\xC3\xB6rld'wrong ! Position is misleading
                      1
Error: Syntax error in PRINT statement at (1)


(This is not a regression as the current version behaves exactly the 
same, however, the use of non-ascii characters will increase and thus it 
makes sense to implement this, e.g. by returning the number of (ascii) 
characters printed via print_wide_char and adding this number to the 
column number.)

> As a consequence of this patch, memory usage to store the source file 
> roughly quadruples.I've not seen a single case where it gives a 
> significant memory increase on the total amount used during 
> compilation higher than 7%
It would be great if a implementaion could be found which reduces the 
memory maximally needed for compilation, which is not a maintenance burden.

> (x86_64-linux). I welcome your comments and reviews. OK to commit?
OK with the manpage change.

Tobias