RFC Wide Characters in I/O

[FX]

>> Yeah; there's probably library functions available for doing a lot  
>> of these conversions back and forth. One problem here might be  
>> that wchar_t is not 32 bits on all platforms, so perhaps we can't  
>> rely on the libc wide char functions?
> If there is something we can use, fine, but it tends to give us  
> less control over what is happening.  We will plan to look into it  
> before we decide whether to roll our own or use someone else's.

We don't have a guaranteed way to deal with these conversions  
available on the system, but we can lift code from libcpp in libcpp/ 
charset.c which has the routines for that:

> /* Conversions between UTF-8 and UTF-16/32 are implemented by custom
>    logic.  This is because a depressing number of systems lack iconv,
>    or have have iconv libraries that do not do these conversions, so
>    we need a fallback implementation for them.  To ensure the fallback
>    doesn't break due to neglect, it is used on all systems. */


The ones we are interested in here are one_utf8_to_cppchar() and  
one_cppchar_to_utf8(), where uchar is "unsigned char" and cppchar_t  
is "gfc_char4_t":

> /* Internal primitives which go from an UTF-8 byte stream to native- 
> endian
>    UTF-32 in a cppchar_t, or vice versa; this avoids an extra  
> marshal/unmarshal
>    operation in several places below.  */



FX

-- 
François-Xavier Coudert
http://www.homepages.ucl.ac.uk/~uccafco/