Re: PATCH: better cpplex.c patch

[Greg McGary <greg at mcgary dot org>]

Jeffrey A Law <law@cygnus.com> writes:

>   In message <msn1hvod19.fsf@mcgary.org>you write:
>   > > Or are we indeed resizing items within the realloc'd buffer too?
>   > 
>   > Yes, we are--at least for now.
> OK.  I missed that.  Sigh.  How gross.

Here's a patch that sets bounds on the token strings allocated from a
token-list namebuf, and expands the bounds of only the token for which
we're expanding the namebuf.  In all other places, token text is
allocated via single calls to malloc or strdup which return the
properly bounded pointer.  We only need special handling for these
token list beasties because they allocate out of a single contiguous
buffer.

Zack, what's the rationale for tokenlists allocating from a single
buffer?  Does this save significant time because you can free a token
list all at once?  Is the savings still enough when you account for
the cost of realloc'ing and relocating token pointers?

2000-09-01  Greg McGary  <greg@mcgary.org>

	* cpplex.c (bounded-ptr.h): New include.
	(_cpp_expand_name_space): Also relocate bounds of token pointers.
	(parse_number, parse_string, save_comment): Set bounds
	of newly minted token.

Index: gcc/cpplex.c
===================================================================
RCS file: /cvs/gcc/egcs/gcc/cpplex.c,v
retrieving revision 1.93
diff -u -p -r1.93 cpplex.c
--- cpplex.c	2000/08/29 18:37:37	1.93
+++ cpplex.c	2000/09/01 18:47:50
@@ -44,6 +44,7 @@ o Correct pastability test for CPP_NAME 
 #include "cpplib.h"
 #include "cpphash.h"
 #include "symcat.h"
+#include "bounded-ptr.h"
 
 static const cpp_token placemarker_token = {0, 0, CPP_PLACEMARKER, 0 UNION_INIT_ZERO};
 static const cpp_token eof_token = {0, 0, CPP_EOF, 0 UNION_INIT_ZERO};
@@ -450,13 +451,19 @@ _cpp_expand_name_space (list, len)
   list->namebuf = (unsigned char *) xrealloc (list->namebuf, list->name_cap);
 
   /* Fix up token text pointers.  */
-  if (list->namebuf != old_namebuf)
+  if (list->namebuf != old_namebuf || __BOUNDED_POINTERS__)
     {
+      long disp = list->namebuf - old_namebuf;
       unsigned int i;
 
       for (i = 0; i < list->tokens_used; i++)
 	if (TOKEN_SPELL (&list->tokens[i]) == SPELL_STRING)
-	  list->tokens[i].val.str.text += (list->namebuf - old_namebuf);
+	  /* The token with zero length is the one on whose behalf
+	     we are expanding the namebuf, so be sure to increment
+	     its high bound to reflect the added space.  No other
+	     tokens should expand their high bounds.  */
+	  RELOCATE_POINTER_INCR_HIGH (list->tokens[i].val.str.text, disp,
+				      (list->tokens[i].val.str.len ? 0 : len));
     }
 }
 
@@ -1049,6 +1056,7 @@ parse_number (pfile, list, name)
  out:
   buffer->cur = cur;
   name->len = namebuf - name->text;
+  SET_BOUNDS (name->text, name->len);
   list->name_used = namebuf - list->namebuf;
 }
 
@@ -1188,6 +1196,7 @@ parse_string (pfile, list, token, termin
  out:
   buffer->cur = cur;
   name->len = namebuf - name->text;
+  SET_BOUNDS (name->text, name->len);
   list->name_used = namebuf - list->namebuf;
 
   if (null_count > 0)
@@ -1216,6 +1225,7 @@ save_comment (list, token, from, len, ty
     _cpp_expand_name_space (list, len);
 
   INIT_TOKEN_STR (list, token);
+  SET_BOUNDS (token->val.str.text, len);
   token->type = CPP_COMMENT;
   token->val.str.len = len;