Re: cpplib: small clean ups

[Zack Weinberg <zack at wolery dot cumb dot org>]

On Sat, May 27, 2000 at 03:04:33PM +0900, Neil Booth wrote:
> This patch just contains a bunch of small cleanups, useful in the
> upcoming macro expander replacement.
> 
> o _cpp_init_toklist now takes a MINIMAL argument, allowing the caller
>   to specify whether to allocate space for a dummy token or not.  Token
>   lists to be stored in the hash table don't want to waste this space.

Could we have symbolic constants for this, like was done for
htab_find_slot's INSERT parameter?

The idea is sound.

> o expand_token_space is now global, and takes a COUNT argument.  Again,
>   this is to prevent wastage for stored macros.

Okay.

> o _cpp_lex_line no longer assumes that it is lexing into the beginning of
>   a token list.  The only bit that assumed this was IS_DIRECTIVE, which
>   checked if the first token was a CPP_HASH.  It now records the initial
>   position on entry, and checks that position for the CPP_HASH instead.
>   Because a function-like macro expansion can cross multiple lines, we
>   have to append to the existing token list when expanding in this
>   case.  The appended logical line could be a directive.
> 
> o Because of the IS_DIRECTIVE change, parse_string2 can no longer tell
>   if it is parsing a directive.  It now takes a boolean argument
>   indicating whether it should allow multiline strings.

Please try lexing into a separate token list when we need to look ahead past
the end of the line.  You can then shuffle it into the normal list when we
decide to proceed scanning the file.  That takes only switching some pointers.  

> o The SPELL_ macros become an enum.

Ok.

> o spell_token takes a "const cpp_token *".

Ok.

> o Pointer fixup on buffer resizing tended to be off-by-one: it missed
>   the current token because the list's token_used hadn't been updated
>   to include it.  This bug in save_comment, and number parsing are
>   fixed.

Ok.

zw