Re: Some (small) c++ compilation profiling data (oprofile)

[Neil Booth <neil at daikokuya dot demon dot co dot uk>]

Zack Weinberg wrote:-

> But that's *not* the correct line/col info.  We should be reporting
> tokens from macro expansion as being located where the macro
> definition was.

Well, yes and no.  I don't think people would be too impressed if
the syntax errors from their 1000-line file were reported to have
been in stdio.h because of some random macro they used.

Really what we want is

In file included from ... blah
whilst expanding macro foo [if appropriate, with location]
and then the error reported as now but pointing to the token in 
the #define.

However, the information about the location of the original
invocation is completely lost at the moment; cpplib would need
to do a bit of book-keeping to retain it.  Ideally the front-end
should just say to print a diagnostic at the location of a given
token, and cpplib would be responsible for the include stack and
reporting the macro position if any.  If we just have a token
pointer, then that is not enough for cpplib to disambiguate it,
since if A is a macro "A A" would return the same pointers twice.

What I am concerned about is how to maintain a smooth stream of
tokens from cpplib to the front end.  I think an array of token
pointers is best (or, slightly worse, an array of tokens).
The front end can then move around in this array freely (as can
cpplib; removing the current restriction of not being able to
back-up more than 1 token if a macro is involved) without the
overhead the current parser has, and with some trivial book-
keeping cpplib can determine the location of the original
macro, if any, provided it's given a location in the array
rather than a pointer to the token.

Now, a cpp token is not enough for the front ends; they want
to add a few bits (e.g. a tree).  So maybe we want something
like

struct fe_token
{
  tree *value;
  unsigned int index;  /* In cpp_token array.  With cpplib's
  			  help this is enough to determine
			  where the original macro was, if
			  any.  The token itself contains
			  the #define position.  */
};

Do you see what I'm trying to get at?  Do you have any better
ideas?

> The only reason we don't do that now - more
> accurately, the only reason why the front end ignores that information -
> is because the front end wasn't/isn't prepared to handle it.  I vote
> we rip out the code that smashes the location information and fix
> whatever breaks.

cpplib doesn't do it either.

> Referring to the C++ front end here, right?  cp/spew.c, or the (much
> less disgusting) stuff in the new parser?  I don't see why we need to
> worry about token locations in order to let the front end say "hang
> onto these, I'm still using them..."

I'm coming from the point of view of keeping token shuffling and the
like to a minimum in both CPP and the new parser.  At present cpplib
is good, but it doesn't lend itself to the new parser doing a good
job.  See above for the best idea I've had (after thinking about it
for a long time, believe me).
 
> Come to think of it, this is exactly the sort of problem that was
> resolved in the compiler proper by garbage collection.  Maybe we
> should have token allocation indirect through a callback, with a
> library-provided default that uses the pools, so that when it's being
> used as part of GCC we can just garbage-collect them all and not worry
> about it.

I think GC is overkill; co-ordinating cpplib and the FE is only a few
lines, and can be done very efficiently.
 
Neil.