Partial fix for libgcj/9802

[Mark Wielaard <mark at klomp dot org>]

Hi,

The following is a partial fix for libgcj/9802 (Bug in surrogate
handling in Unicode to UTF-8  conversion). This only fixes the case for
UTF-8 surrogates but as James Clark explains this can also occur in
other multibyte encodings.

In principle the other encoders can also be rewritten to use the new
bytes_todo field to indicate that more output is available. But I am
hoping that converting the encoders to the new java.nio.charset
framework will eliminate this problem since it has explicit support for
this (see CoderResult, Jesse Rosenstock will certainly correct me if I
am wrong). But I do not expect that we can finish that work for 3.3, so
just fixing it now for UTF-8 seems worthwhile.

I also added the testcase that James Clark made to Mauve and it passes
with this patch. Since non of the other encoders use the bytes_todo
field this does not impact any other encoders, just UTF-8.

2002-02-22  Mark Wielaard  <mark at klomp dot org>

        Partial fix for PR libgcj/8738:
        * gnu/gcj/convert/UnicodeToBytes.java (bytes_todo): New field.
        (done): Reset bytes_todo field.
        * gnu/gcj/convert/Output_UTF8.java (bytes_todo): Removed field.
        (write): Always decrease avail when count is increased.
        * java/lang/natString.cc (getByes): Check converter->bytes_todo.

OK for branch and mainline?

Cheers,

Mark

Index: gnu/gcj/convert/Output_UTF8.java
===================================================================
RCS file: /cvs/gcc/gcc/libjava/gnu/gcj/convert/Output_UTF8.java,v
retrieving revision 1.6
diff -u -r1.6 Output_UTF8.java
--- gnu/gcj/convert/Output_UTF8.java	8 Aug 2000 17:35:32 -0000	1.6
+++ gnu/gcj/convert/Output_UTF8.java	22 Feb 2003 17:40:28 -0000
@@ -1,4 +1,4 @@
-/* Copyright (C) 1999, 2000  Free Software Foundation
+/* Copyright (C) 1999, 2000, 2003  Free Software Foundation
 
    This file is part of libgcj.
 
@@ -27,8 +27,6 @@
   char hi_part;
   // Value of incomplete character.
   int value;
-  // Number of continuation bytes still to emit.
-  int bytes_todo;
 
   public int write (char[] inbuffer, int inpos, int inlength)
   {
@@ -104,6 +102,7 @@
 	      {
 		value = (hi_part - 0xD800) * 0x400 + (ch - 0xDC00) + 0x10000;
 		buf[count++] = (byte) (0xF0 | (value >> 18));
+		avail--;
 		bytes_todo = 3;
 		hi_part = 0;
 	      }
Index: gnu/gcj/convert/UnicodeToBytes.java
===================================================================
RCS file: /cvs/gcc/gcc/libjava/gnu/gcj/convert/UnicodeToBytes.java,v
retrieving revision 1.9
diff -u -r1.9 UnicodeToBytes.java
--- gnu/gcj/convert/UnicodeToBytes.java	18 Aug 2001 03:56:01 -0000	1.9
+++ gnu/gcj/convert/UnicodeToBytes.java	22 Feb 2003 17:40:28 -0000
@@ -1,4 +1,4 @@
-/* Copyright (C) 1999, 2000, 2001  Free Software Foundation
+/* Copyright (C) 1999, 2000, 2001, 2003  Free Software Foundation
 
    This file is part of libgcj.
 
@@ -15,6 +15,9 @@
   public byte[] buf;
   public int count;
 
+  /** Remaining bytes that couldn't yet be written to buf. */
+  public int bytes_todo;
+
   // The name of the default encoding.
   static String defaultEncoding;
 
@@ -157,6 +160,7 @@
       {
 	this.buf = null;
 	this.count = 0;
+	this.bytes_todo = 0;
 
 	encoderCache[currCachePos] = this;
 	currCachePos = (currCachePos + 1) % CACHE_SIZE;
Index: java/lang/natString.cc
===================================================================
RCS file: /cvs/gcc/gcc/libjava/java/lang/natString.cc,v
retrieving revision 1.27
diff -u -r1.27 natString.cc
--- java/lang/natString.cc	13 Jun 2002 18:16:26 -0000	1.27
+++ java/lang/natString.cc	22 Feb 2003 17:40:28 -0000
@@ -1,6 +1,6 @@
 // natString.cc - Implementation of java.lang.String native methods.
 
-/* Copyright (C) 1998, 1999, 2000, 2001, 2002  Free Software Foundation
+/* Copyright (C) 1998, 1999, 2000, 2001, 2002, 2003  Free Software Foundation
 
    This file is part of libgcj.
 
@@ -586,7 +586,7 @@
   jint offset = 0;
   gnu::gcj::convert::UnicodeToBytes *converter
     = gnu::gcj::convert::UnicodeToBytes::getEncoder(enc);
-  while (todo > 0)
+  while (todo > 0 || converter->bytes_todo > 0)
     {
       converter->setOutput(buffer, bufpos);
       int converted = converter->write(this, offset, todo, NULL);