Re: cpplib: new lexer

[Zack Weinberg <zack at wolery dot cumb dot org>]

On Sun, Apr 23, 2000 at 07:44:31PM +0900, Neil Booth wrote:
> This diff contains a new lexer for cpplib.  It has various benefits
> over the current one, including:
[stuff...]

> The code is #if-ed out at present, since hooking it up to everything
> else will be a fairly big job.  Do you want me to put it in a separate
> file, e.g. cpplexline.c, Zack?  I've included a few helper functions
> that I used for testing, like _cpp_lex_file and _cpp_output_list,
> as they're useful but will probably be removed in the long run.
> 
> Does anyone want a big comment at the top roughing out how it works?

Okay.  There are some minor issues with the patch but since you are
going to be out of touch next week, we should probably put it in and
then work on polishing it up.

I would rather you did not create another file.  Most of the existing
cpplex.c will disappear after we turn this on.

Yes, please write a big comment at the top explaining how it works.

Some of the functions in your code have already been merged.  Please
figure out which those are and don't duplicate them.  If you need
different behavior, either change the code that's using it to match,
or #ifdef out the new behavior and I'll fix it later.  Also, please go
through everywhere and replace HSPACE_BEFORE with PREV_WHITESPACE.

And please don't create a STRINGIZE token - you wanted it for macro
expansion, but we can handle that with flags on the MACRO_ARG token.

Once you've made these changes you may commit the patch.  Remember to
send the revised patch to the list.

zw