This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: GCC trunk SPEC2000 performance
On Fri, 21 Jun 2002, Jan Hubicka wrote:
> > It could. Would you like to suggest a patch (perhaps to
> > contrib/regression/btest-gcc.sh), or a test procedure? That would
> > help the discussion, I think.
> >
>
You could always run SPEC after bootstrapping, but that is
already being taken care of elsewhere. Both SPEC95 and SPEC2000
are run daily. Besides, at 4 hours for SPEC95 and 10-12 hours
for SPEC2000, that is an extremely expensive proposition for the
automatic bootstrap tester.
> It may be interesting to use some statistics suite to postprocess the
> results from Andreas/Diegos testers, find the gaps and put the links
> in logs+notify the authors. I tried to do this by hand for a while and
>
Agreed. Furthermore, SPEC always exhibits a 1-5% variation from
run to run. To identify a true regression, you need to have a
valley that lasts for at least 2-3 runs. That's when you go to
the first day of the valley and start looking at the diffs.
> We may think about using simplier testsuite targeted for various
> optimizations. Mark Lehnman and I attempted to put something together
> at benchmarks subdirectory, but that is not maintained for a while.
>
There's always nullstone (http://www.nullstone.com), but I don't
have a license to it.
Diego.