Re: PCH, and more generally C++ parser performance

[Stan Shebs <shebs at apple dot com>]

Some very interesting analysis!  Just a couple comments:

Zack Weinberg wrote:
> 
> Third, we could not generate so much data in the first place.  Most of
> that 45 megs is saved tree representations of inline functions.  Most
> of those inline functions are never used.  If we could avoid parsing
> them in the first place, wouldn't that be nice?  Here's where you're
> all thinking precompiled headers.

That probably explains the big discrepancy between C and C++ performance.
Volumewise, only a small percentage of the synthetic example is class
definitions, but it jumped the compile time by some 50-60%.  Of course,
this will be more of an issue for systems that use more C++ headers; the
basic Mac API is 98% C.  On Macs, the primary source of C++ headers is
Metrowerks' PowerPlant, which is sizeable, but nowhere near the base API.

> Apple inherited a remarkably clever PCH implementation from NeXT.  It
> can figure out which functions the compiler actually needs to see, and
> present it with only those.  Alas, it only works with C; but Stan
> reports that for the C version of the above example, compiler runtime
> with PCH is less than a second.

Yeah, it's amazing how fast GCC is when it doesn't have to parse thousands
and thousands of unreferenced enums and prototypes... :-)

Incidentally, I've been on a crash project to add C++ to cpp-precomp
(as we call it), am learning way more about dark corners of C++ syntax
than I really wanted to know. :-)  Also - but keep in mind this is not a
promise - I'm getting some favorable response to the idea of freeing the
sources to cpp-precomp.  Although it's not reasonable to try to include
any of it directly into GCC (it's written in C++, for one thing), it
should be useful as a source of ideas on how to make precompiled headers
work.

> (Part of that might be the simpler C
> front end, though.)  We can do something similar without necessarily
> implementing PCH.  The C++ front end already has logic to delay
> parsing inline functions until convenient.  It is used only to work
> around some of the horrors of parsing C++ at the moment, but it could
> be applied to every inline function, and they could be delayed until
> actually required instead of just until the context for parsing them
> is better.  The unparsed structure is somewhat smaller, and much
> simpler, than the tree it becomes.

That sounds like a good idea.  Token streams are actually very compact,
you can get down to an average of little more than 1 byte/token, which
is smaller than either source or tree structure.  cpp-precomp basically
works by having various tables indexing the sea of tokens, and pulling
out ranges of tokens for further processing.

Stan