Re: cpplib: line-at-a-time lexing

[Zack Weinberg <zack at codesourcery dot com>]

On Mon, Sep 09, 2002 at 03:08:58PM +0100, Neil Booth wrote:
> This is about the bare minimum necessary to get cpplib to lex
> lines at a time, rather than the current method of lexing the
> next token only when it's needed.

Neat.

> This patch is good enough to bootstrap c, objc and c++ without
> testsuite regressions.  It is still buggy, in that resizing an
> internal token array at the wrong time can leave pointers
> dangling.  The default array sizes are such that this is a rare
> occurrence.

Can you construct a test case that reliably hits the bug?

> However, this makes me reluctant to commit it to the BIB branch; so
> unless you think it should go in anyway I'm just posting it as an
> FYI, Zack.

Yeah, let's keep it out until it's good and solid.

> no longer complain that "foo" is poisioned since it wasn't when we
> lexed it.  Also, we used to permit re-poisoning a poisoned identifier,
> this now complains about use of a poisoned identifier.  I don't think
> either of these is a great loss, but the latter can be recreated with
> a bit of effort if it's deemed worthwhile (the problem is context
> sensitivity: to not warn about being re-poisoned, we need to know
> if we're in a #pragma poison when lexing, so this would need a scan
> of already-lexed tokens that I've not written.  There is a similar issue
> with __VA_ARGS__, but I've written the code for that).  The former
> is not really fixable, but easily worked around by documenting
> that #pragma GCC poison only applies from the subsequent line.

I think these constraints are fine.

I happen to think that the C99 requirement that __VA_ARGS__ appear
only in a replacement list of a variadic macro is utterly bogus, and
this is a nice demonstration of why.  I wonder, if we just stuck in
code in all the relevant places to prevent it being used as a name of
anything, would that cover enough bases to satisfy the diagnostic
requirement?

zw