This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: Proposal for 2 Byte Unicode implementation in gcc and glibc
- To: linux-utf8 at nl dot linux dot org, haible at ilog dot fr
- Subject: Re: Proposal for 2 Byte Unicode implementation in gcc and glibc
- From: Werner LEMBERG <werner at m17n dot org>
- Date: Sat, 05 Aug 2000 03:47:17 +0200 (CEST)
- Cc: sap-list at redhat dot com, gcc at gcc dot gnu dot org, wilhelm dot nuesser at sap dot com, hans-christoph dot rohland at sap dot com
- References: <816D93CCC927D31188570008C75D1DE1011A0BDF@dbwdfx1a.wdf.sap-ag.de><14731.1243.115781.953710@honolulu.ilog.fr>
- Reply-To: Werner LEMBERG <wl at gnu dot org>
> > For a number of languages, the UTF-8 representation saves some
> > storage when compared with UTF-16, but for Asian characters UTF-8
> > requires 50% more storage than UTF-16.
>
> Yes, it does. And for English and German UTF-16 requires 100% more
> storage than UTF-8.
You can use SCSU to compress your data. It works with short strings
also (which is not true for generic compression algorithms like LZW).
The Technical Report #6 (http://www.unicode.org/unicode/reports/tr6/)
gives the following examples:
UTF-16 German: 9 chars (18 Bytes) -> SCSU 9 Bytes
Russian: 6 chars (12 Bytes) -> 7 Bytes
Japanese: 116 chars (232 Bytes) -> 178 Bytes
Werner