Re: Patch to improve cpplib's C++ support

[Zack Weinberg <zack at rabi dot columbia dot edu>]

On 11 Jul 1999 18:33:23 -0700, Jason Merrill wrote:
>>>>>> Jason Merrill <jason@cygnus.com> writes:
>
>>>>>> Jason Merrill <jason@cygnus.com> writes:
> >> cpp_unget is necessary because parsing sometimes looks ahead at the next
> >> token to determine how to proceed.  Normally, this is fine, but when we're
> >> switching to a different input stream, we need to put the lookahead token
> >> back into the buffer where we found it so that it will be there when we
> >> return to reading from that file.
>
> >> Actually, it occurs to me that it's the lexer, not the parser, that's
> >> looking ahead in this case, and that I can probably avoid that by taking
> >> better advantage of our pre-knowledge of token lengths.
>
> > Nope, I was wrong.  We need to look ahead at the next token in order to
> > support type attributes.
>
>No, I was right the first time; I've fixed g++ so cpp_unget isn't needed.
>I'll investigate doing the same for redirected_input_p.

Now I see what they are for, I don't have a real problem with either
feature.

However, I'm working on a revised API to cpplib that is going to behave
differently in both the affected areas.  The big difference is that you make
one function call and get handed an entire line split up into tokens.  You
then can walk through it at your leisure.  If you stack another input buffer
in the middle of parsing the list, it'll still be there when you come back
(this is needed anyway for macro expansion).  So cpp_unget becomes a pointer
decrement in the front end.  On the flip side, you can't back up by part of
a token.  cpp_unget as it was in your patch would have let you do that.

As for redirected_input_p, the plan is to make caller pop the buffer all the
time (except for macro buffers).

zw