Re: Pre-compiled headers

["Lassi A. Tuura" <lassi dot tuura at cern dot ch>]

> So, Cygnus has contracted me to implement precompiled headers.

Excellent!  Brilliant!  Can't wait to try your work out...

> 1. The one huge cache file arrangement.  You have one mega-header
>    file that includes all the system headers and makes all the
>    internal definitions you need.  You run that through a cruncher to
>    produce the precompiled header, and #include the precompiled
>    version.

I am for this strategy, though with a twist.  What I would ask is to
allow precompilation itself and use of it to follow this pattern:
  - put all relevant #includes in all.h
  - generate all.pch
  - run subsequent compiles with all.pch

The difference here is to allow `all.pch' to be given on command line;
no source files should need to include `all.h'.

This has turned out to be the only solution that works in practise for
us, out of a few different schemes; most of them have died straight
away.  We have relatively large code base (some hunderds of MB of C++
and FORTRAN) of ~150 packages in a tree structure.  The build time is
somewhere around 2 hours in a four-way parallel build on a fast linux
server.  For the platforms we support (several, native compilers), the
build trees end up around 3-4 GB (source, all build results, installed
files) before we clean it up; the resulting tree is currently a tad
over 2 GB.  But then, we are only getting started :-) The code base
will probably grow to three times its current size or more.

Here are some issues that may help you understand our situation:

1) Our build tree is on a networked file system (AFS).  It uses local
   cache (~200 MB/workstation) and the network is fast, but our build
   tree is so large that it easily swamps any cache.  Heck, one C++
   library of five simple but STL-heavy source files can be 50+ MB
   (anyone for compressing debugging information?  I can't wait for
   -fsquangle...).  FWIW, building on a local disk is a factor of 4-10
   faster, but we can't do that systematically.

   We do have rather generous amounts of disk space, but given the
   networking, file size does matter. It is OK to create a .pch of
   some megabytes per package, but any scheme that wants many such
   files per package we will be much less likely to use.

2) #include "all.h" is out.  Two main reasons: too many compilers do
   not properly support (read: real working no-bugs implementation)
   precompiled headers, and sticking in the "all.h" will slow such
   compiles to a crawl (including gcc today).  Second: we have a
   rather large number of, umm, unsophisticated developers, mostly
   scientists scattered around the world.  For a precompiled header
   scheme to be useful for us, It Just Has To Work(tm).  We won't be
   able to teach everybody the mysteries of umpteen different
   precompiled header schemes I am afraid.

   From our point of view it is best to hide the functionality in our
   build system.  It has limited knowledge about each package, so it
   can really only provide is a blanket `all.h' for system headers
   (C++ standard library is already a great improvement).  Since the
   build system completely determines how to invoke the compiler, it
   can just precompile the system headers in each package and throw in
   the option for good measure to speed things up.  If the precompiled
   header scheme is robust enough, who knows, it might be possible to
   share a single precompiled header across all packages.

3) We have lots of code.  If a precondition to being able to exploit
   the system is to rearrange #include statements to some common order
   or to collect them into an `all.h' in each package, it will take
   much longer for us to be able to use the scheme -- possibly
   infinitely long.  IMHO separately compiled and given precompiled
   header is much more friendly for existing code bases -- and even
   for future coding.  (Our style also guides recommend that header
   files include minimal other headers, not only for speed but also
   for documentation and to avoid inadvertent dependencies.  I guess
   this would have to be reconsidered in the light of precompiled
   header schemes, but I would much like the precompiled header scheme
   to also take such views into account.)

4) I've seen too many systems that require developer intervention,
   cleaning up files, failures to notice file changes properly,
   failures to notice differences in compiler options that render
   caches of various sorts invalid, failures of noticing when same
   source file is compiled several times in different ways, compiler
   errors arising from corrupted pch files, compilers getting confused
   by the precompiled header, incorrect line/column information, and
   many more.  What I am trying to say is that I would much rather
   take a simple system that works rock solid than a sophisticated
   system that works only in 90% of cases or requires a lot of
   attention.

>    Disadvantages: Doesn't apply easily to any other project structure.

Does my proposal to supply all.pch on command line change that?

I suppose one question you want to ask is what exactly are you trying
to achieve?  Are you trying to make big mega-size builds go fast,
individual developer builds, or what?  The schemes I have seen seem
mostly designed for the latter, leaving our types of environments in
the cold -- but ours could really use some serious speed-up!  Consider
also folks building releases of all GNU software for CD or local
network distribution.  I doubt they are enthusiastic to fiddle with
the code, but would not mind being able to speed up the builds just by
using magic CXXFLAGS for `configure'.  Or other existing projects like
Linux and Gnome that are not designed for precompiled headers.

If you decide to go for the second scheme, please try to research
existing similar schemes.  It would be a shame if gcc required headers
in certain order to be able to use a precompiled header, and many
other compilers required them in a much different order.  I know
compatibility with non-GNU systems is not the highest priority, but I
am sure you agree it doesn't make sense to be different for just the
sake of being different :-)

Well, that's my 0.02 (euro)cents.  Hope they are useful.

Cheers and many thanks for your excellent work,
//lat
-- 
I've seen professional jurors.  I prefer road kill.  --Mike O'Brien