This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Problem with PFE approach [Was: Faster compilation speed]
- From: "Timothy J. Wood" <tjw at omnigroup dot com>
- To: Mike Stump <mrs at apple dot com>
- Cc: gcc at gcc dot gnu dot org
- Date: Fri, 16 Aug 2002 13:31:26 -0700
- Subject: Problem with PFE approach [Was: Faster compilation speed]
So, another point in favor of discarding the concept of 'statically
precompilation' based on a problem I just ran into with PFE under
10.2...
I'm emulating some of the Win32 API for porting games to Mac OS X.
Win32 has a macro like this:
#ifndef INITGUID
#define DEFINE_GUID(name, l, w1, w2, b1, b2, b3, b4, b5, b6, b7, b8) \
EXTERN_C const GUID FAR name
#else
#define DEFINE_GUID(name, l, w1, w2, b1, b2, b3, b4, b5, b6, b7, b8) \
EXTERN_C const GUID name \
= { l, w1, w2, { b1, b2, b3, b4, b5, b6, b7, b8 }
}
#endif // INITGUID
If this gets stuck in a PFE and the PFE is applied as a prefix header
(the only way it can be done right now), then the file being compiled
cannot make its own decision about whether INITGUID should be defined
or not.
Clearly there are ways around this, but the current approach makes
the compiler produce different output based on whether PFE is on or
not. I consider this a bug.
This would not be a problem with an automatic precompiler that
remembered facts and didn't use the prefix header hack.
Are there problems with what I describe below or are people just
avoiding commenting on this since it is too hard to implement? :)
-tim
On Friday, August 9, 2002, at 02:58 PM, Timothy J. Wood wrote:
2) This one is rather crazy and would involve huge amounts of work
probably....
a) Toss some or all of your PFE code in the bin (yikes!)
b) Build a precompile server that the compiler can attach to and
request precompiled headers (give a path and set of -D flags or
whatever other state is needed to uniquely identify the precompile
output). Requests would be satisfied via shared memory (yes,
non-portable, so this whole mechanism will only work on modern
machines).
c) Inside the server, keep parsed representations of all headers
that have been imported and the -D state used when parsing the
headers. As new headers are parsed, they should be able to **layer**
on top of existing parsed headers (so there should only be one parsed
version of std::string). This avoids the confining requirement that
you have one big master precompiled header.
d) Details about concurrency, security, locating the server, and so
on left as an exercise for the reader.
The main advantage here is that people would get fast compiles
WITHOUT having to tune their single PFE header. Additionally, more
headers would get precompiled than would otherwise, yielding faster
builds. If they layering is done correctly, the memory usage of the
entire system could be lower (since if you have two projects to build,
both of which import STL, there would be only one precompiled version
of STL).
At the start of a build, a special 'check filesystem' command could
be sent to the server to have it do a one-time check of timestamps of
headers files. Assuming the timestamps haven't changed, the
precompiled headers could be kept across builds!
Naturally doing a 'clean' build from the IDE option would need to be
able to flush and probably shut down the server since it is inevitable
that there will be bugs that will corrupt the precomp database :(
#2 could really take many forms. The key idea is that having a
single PFE file is non-optimal. Developers should not have to spend
time tuning such a file to get the best compile time. The compiler
and IDE should handle all these details by default. Having the
developer involved here just leads to extra (ongoing!) work for the
developer and a sub-optimal set of precompiled headers.
Your goal should be to have the developer open their project and
have it build 6x faster (instead of requiring the developer to do a
several hours of tweaking on their PFE file to get the best
performance -- and then having to keep it up to date over the life of
their project).
3) This is possibly even harder... Keep track of what facts in a
header each source file cared about (macro values defined or
undefined, structure layout, function signature, etc, etc, etc). If a
header changes, have the precompile server keep track of the facts
that have changed and then only rebuild source files that care about
those changes (assuming the source file itself hasn't compiled). This
could get really ugly since you'd potentially keep track of multiple
fact timestamps (consider if a build fails or is aborted so some files
got updated for the current state of a header and some didn't).
Extra bonus points for doing this on a lower granularity basis
(i.e., don't recompile a function if it wouldn't produce different
output). This would clearly be very hard and a large departure from
the current state of affairs :)
Anyway, I think the biggest improvements lie in moving away from the
current batch compile philosophy mandated by the command line tools.
Instead, the command line tools should be a front end onto a much more
powerful persistent compile server.
(Hey, you asked for ideas and said it was OK if they were hard :)
-tim