preprocessor/14438

[Alexandre Oliva]

The reason why the patch had introduced a regression was that
pfile->cur_token pointed to uninitialized memory, one past the last
initialized token.  _cpp_lex_direct() doesn't skip to the next line
because we're processing a directive, so it just sets result->type to
CPP_EOF, along with its correct location, after incrementing
pfile->cur_token past result, without any form of initialization to
this value.  In fact, I'm a bit concerned as to what might happen if
we were to call _cpp_lex_direct() an unlimited number of times while
processing a directive; I don't see anything that would prevent
cur_token from running past the memory area reserved for it.

I suppose one way to fix this issue would be to decrement cur_token
again when returning in the CPP_EOF case, such that we'd return the
same EOF token over and over; another would be to just refrain from
dereferencing cur_token after reading an EOF token, but how would we
tell?  And, more importantly, why would we want to?  Clearly we don't
need it in the case I thought we did when I added the code I just
removed; I don't think we need it elsewhere.

Is this analysis enough for us to close this bug report now?

-- 
Alexandre Oliva             http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer   aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist  oliva@{lsd.ic.unicamp.br, gnu.org}