Re: fix kennerism

[kenner at vlsi1 dot ultra dot nyu dot edu (Richard Kenner)]

The formalism helped a lot, thanks.  It seems we're talking about almost
the same thing, except for two comments:
    
    Take a tree T that is the FSF GCC tree as of time t, with no patches
    applied.  Test T.  Test T+P where P is a single patch.

The purpose of the test on T seems marginal to me.  The costs of doing it is
not only doubling the test time but of having test harnasses that operate on
more than one directory.  Doing that creates a real risk of losing track of
what's in each directory and introducing procedural errors.

The only thing you gain from the test on T is if the T+P test produces worse
results that the last such test.  In that case, it's indeed true that having
the results of T around help diagnose the problem, but there are other ways
of doing it, for example running T *then*.  (Yes, there is the tiny risk
that the step forward on the tree fixed some test that used to fail and P
broke that same test and it'll go unnoticed, but we can ignore that.)

The simplest procedural way of working is to have a tree that is a copy
of the FSF tree plus local patches where the intent is to test and install
those local patches as soon as possible.  In that situation, all CVS commands
are done in that directory, all testing is done from the source base
in that directory, and all development is done in that directory.  There's
little potential for confusion about what is where in such a case.

If you start having multiple source trees, there is always the potential for
confusion over what is where.  People who are doing development where they
have local changes that cannot be committed for some period of time have no
choice but to buy into this complexity, but I don't see that the gain people
who aren't in this situation outweighs the potential for error.

    A test run, for multiple patches, should most safely test the trees
    (T, T+P1, T+P2, ...) or (T, T+P1, T+P1+P2, T+P1+P2+P3, ...) if they
    are to be installed together.  One patch might fix a problem, which
    another patch then causes to appear again; or one might cause a
    problem, which another covers up.  But separate testing is more
    important where patches are being submitted for approval than when
    they will be installed together immediately afterwards.

Right.  However, I think people should be able to submit a bunch of
patches and say "I tested these all together; if you're only going to
approve a subset of them, let me know and I'll test that subset".