This is the mail archive of the
gcc-patches@gcc.gnu.org
mailing list for the GCC project.
Re: [CFARM] Script for automatic checking of patches
On 03/07/07, Zdenek Dvorak <rakdver@kam.mff.cuni.cz> wrote:
Hello,
> > > I'm still quite annoyed by the fact that when new fails are inserted
> > > in trunk, there is no way to filter out these new fails from the fails
> > > due to the tested patch. One possible solution is to schedule every
> > > now and then a bootstrap of a pristine tree to reset the set of
> > > passing tests, but this can be done outside the tester.
> >
> > what we do in our tester is that we build gcc twice (once without the
> > patch, once with) and compare the results. It doubles the time of the
> > testing, but the results are more reliable than with the scheme you
> > propose.
>
> But if you test N patches against the same revision, the time will be
> N+1, which is a lot for N=1 but not bad at N=10. The compile farm is fast
> enough that you can do so before GCC changes a lot.
except that in many cases you want to prescribe a different revision
-- if you are testing a patch for branch
-- if you want to be sure that the change is correct with the most
current sources
If you are testing a patch for a branch the whole issue is more
complicated, since you need a specific svn switch command. (the script
could take also a "branch: BRANCH" tag that is passed to "svn switch
svn://gcc.gnu.org/svn/branches/BRANCH")
Putting aside that special case, one possible approach is to have a
computer just doing svn up && bootstrap + full check of pristine
trunk, then saving the results according to the revision (a bunch of
sum and log files, one could gzip them if needed at all). Then, the
patch testing script will look which is the latest (successful)
bootstrap+fullcheck available, svn up to that revision, bootstrap+test
the patch and compare with the pristine results. This will have the
additional benefit that if bootstrap is broken in mainline, the patch
testing script will not jump to the broken revision.
This could even be done in a single computer, in parallel if there is
enough memory, or every X number of patches (or patch testing queue is
empty).
Anyway, testing twice (with and without the patch) is just the easiest
and simplest way. It could be improved incrementally if the system is
actually so much used that the queue of patches is continually
increasing. I think the real question at this moment is: what are the
minimum requirements to make the script interesting to (sporadic and
long-time) contributors?
Cheers,
Manuel.