[gcjx] Patch: FYI: another round of lexer tweaks

[Tom Tromey <tromey at redhat dot com>]

I'm checking this in on the gcjx branch.

Not doing too good today, my lexer fix caused a regression in another
set of jacks tests (and my next patch is also for a regression I
introduced today.  Bleah).  This fixes those, and this time was tested
against all of jacks, not just one section.

Tom

Index: ChangeLog
from  Tom Tromey  <tromey@redhat.com>
	* source/lex.cc (lexer): Updated.
	(get): Rewrote again.
	* source/lex.hh (lexer::was_return): New field.

Index: source/lex.cc
===================================================================
RCS file: /cvs/gcc/gcc/gcjx/source/Attic/lex.cc,v
retrieving revision 1.1.2.2
diff -u -r1.1.2.2 lex.cc
--- source/lex.cc 15 Jan 2005 22:01:08 -0000 1.1.2.2
+++ source/lex.cc 15 Jan 2005 23:09:13 -0000
@@ -35,6 +35,7 @@
   : input_filter (source),
     backslash_count (0),
     unget_value (UNICODE_W_NONE),
+    was_return (false),
     cooked_unget_value (UNICODE_W_NONE),
     scratch (NULL),
     scratch_size (0),
@@ -293,8 +294,8 @@
   return UNICODE_BACKSLASH;
 }
 
-// Get a single character, having already processed \u sequences and
-// collapsing line terminators.
+// Get a single character, having already processed \u sequences.
+// Collapse line terminators.
 unicode_w_t
 lexer::get ()
 {
@@ -307,16 +308,18 @@
   else
     c = read_handling_escapes ();
 
-  if (c == UNICODE_CARRIAGE_RETURN)
+  if (was_return && c == UNICODE_LINE_FEED)
     {
+      // Saw \r\n, so read again.
       c = read_handling_escapes ();
-      if (c != UNICODE_LINE_FEED)
-	{
-	  // Saw \r followed by something else, so unget and return the
-	  // line terminator that the rest of the lexer understands.
-	  unget (c);
-	  c = UNICODE_LINE_FEED;
-	}
+    }
+
+  was_return = (c == UNICODE_CARRIAGE_RETURN);
+  if (was_return)
+    {
+      // Return the kind of newline that the rest of the lexer
+      // understands.
+      c = UNICODE_LINE_FEED;
     }
 
   if (c == UNICODE_LINE_FEED)
Index: source/lex.hh
===================================================================
RCS file: /cvs/gcc/gcc/gcjx/source/Attic/lex.hh,v
retrieving revision 1.1.2.1
diff -u -r1.1.2.1 lex.hh
--- source/lex.hh 13 Jan 2005 03:18:37 -0000 1.1.2.1
+++ source/lex.hh 15 Jan 2005 23:09:13 -0000
@@ -41,6 +41,9 @@
   // value is only used by escape processing.
   unicode_w_t unget_value;
 
+  // True if we previously saw a \r.  Use by get().
+  bool was_return;
+
   // There's a second layer of unget handling.  This field is used by
   // the more common get()/unget()/peek() set of methods.
   unicode_w_t cooked_unget_value;