This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: Performance regression testing?
Joe Buck wrote:
> It would be possible to detect performance regression after fact, but
> soon enough to look at reverting patches. For example, given multiple
> machines doing SPEC benchmark runs every night, the alarm could be raised
> if a significant performance regression is detected.
Right; I think we do some of that at present.
I was hoping that having it there when people did test runs would change
the psychology; instead of having already checked in a patch, which
we're then looking to revert, we'd be making ourselves aware of
performance impact before check-in, even for patches that we don't
expect to have performance impact. (For major new optimizations, we
already expect people to do some benchmarking.)
But, yes, this is a definite alternative: we could further automate the
SPEC testers, or try to set up more of them.
>>As a strawman, perhaps we could add a small integer program (bzip?) and
>>a small floating-point program to the testsuite, and have DejaGNU print
>>out the number of iterations of each that run in 10 seconds.
>
> Would that really catch much?
I really don't know. That's why it's a strawman. :-)
--
Mark Mitchell
CodeSourcery, LLC
mark@codesourcery.com
(916) 791-8304