This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: compile time testsuite [Re: alias slowdown?]
On Tue, 2006-11-21 at 15:26 +0100, Richard Guenther wrote:
> On 11/21/06, Benjamin Kosnik <bkoz@redhat.com> wrote:
> >
> > So, I consider public monitoring complementary, but not something that
> > obviates the real need for a consensus on what exactly are the things
> > that are going to measured for compile-time regressions.
>
> One problem with in-repository testing is that for compile-time performance
> or memory usage you depend on a baseline or known good value. At least
I *always* get a baseline for the source I am using first (I want the
build anyway to verify it bootstraps and what testsuite regressions
there are before my cod is applied).
Then do all my work and compare against that baseline I established when
I checked the code out. I have to compare the new source on my machine
against the original source on my machine, or my comparison is flawed.
> to automatically get a "FAIL" result here. With recommended two runs,
> one without, one with a patch we could do a post-processing script that
> checks for regressions (hopefully not too often trapping on noise...).
>
> (This is also one thing that makes the libstdc++ performance testsuite less
> useful)
The web page is good for monitoring what everyone together has done, and
tracking the result. It doesn't help me to determine whether *my*
change is faster or slower. And thats what I need to know as a developer
working on a pass. I especially nee dto know if the pass I am working on
is faster or slower. For that I need to be able to compare individual
pass times as well.
I think we absolutely need a make compile-time, and a script that will
paw through it and find the times for specific passes as requested.
Currently since its just for me, I do it by hand on my 4 or 5 testcases.
Andrew