Re: -traditional comment elimination (yuk...)

[Dave Brolley <brolley at redhat dot com>]

Neil Booth wrote:

> > Therefore, I see no particular reason why we can't just shut off -C
> > when processing directives other than #define.  That means the
> > comments disappear, but the rest of the line's going to disappear too,
> > so the comments are less than helpful.
>
> My feelings are moving towards shutting off -C for preprocessing
> directives too.

Yes -- this is the correct behaviour and, in fact, is the reason why
comment pasting in macros in traditional preprocessors works.

#define foobar 1
#define cat(a,b) a/**/b

In a traditional preprocessor, cat (foo,bar) produces 1 *everywhere* (open
text and on directives) because the preprocessor removes the comment and
then rescans the substituted text. This is an example where the two pass
approach will still not be adequate since the expanded macro may never get
re-lexed by the compiler.

#if cat(foo,bar)
this_code_should_get _compiled();
#endif


> > We still arguably have a problem with something like
> >
> > #def/**/ine foo bar
>
> In fact we actually get that right at present, by luck rather than
> design.

This is no problem with the two pass approach since it should not work
anyway.

> > in -C -traditional mode only.  But look at this:
> >
> > % cat test.c
> > #def/**/ine foo bar
> > foo
> > % /lib/cpp test.c
> > # 1 "test.c"
> > test.c: 1: undefined control
>
> But that *is* a K&R compiler's behaviour.

Correct.

> My understand from Dave's post is that with -traditional, the effect
> of comments on the preprocessor and front end are different, because
> K&R compilers had a (conceptually, at least) separate preprocessor.
>
> To the preprocessor, a traditional comment acts as a separator, just
> like PREV_WHITESPACE does for us now.  The difference is that no space
> is actually piped into the K&R front end, which does it's own lexing.
> Thus the front end sees pasted tokens, and the preprocessor sees
> separate tokens.  Right Dave?

Correct.

> In the light of the above, if my understanding is correct, I am now
> convinced of the right way forward, and it won't affect the current
> lexer too much.
>
> 1) Just like a true K&R compiler, we make -traditional go through two
> passes, via a temporary file just like Dave suggested.  This means we
> get the same tokenisation as a true K&R compiler would.
>
> 2) We need an extra flag TRAD_COMMENT for tokens to indicate a
> traditional comment, to get the effect of not outputting whitespace.

Why not just test for the -traditional option when deciding whether to
replace a comment by a space?

To summarize: I believe that the two pass approach for -traditional will
get most cases right, since that's exactly what a traditional (K&R)
preprocessor/compiler system did. *However*, there are still some
-traditional effects to be dealt with in the preprocessor which are the
result of the fact that -traditional preprocessors were also *internally*
text-based whereas cpplib is token based. See my example of the 'cat' macro
above.

Dave