This is the mail archive of the gcc@gcc.gnu.org mailing list for the GCC project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: compile time testsuite [Re: alias slowdown?]


> One problem with in-repository testing is that for compile-time performance
> or memory usage you depend on a baseline or known good value.  At least
> to automatically get a "FAIL" result here.  With recommended two runs,
> one without, one with a patch we could do a post-processing script that
> checks for regressions (hopefully not too often trapping on noise...).

Yes. For anything that is machine-dependent we'll have to depend on
baselines from stable releases or previous runs. I don't see a way
around this. (Note this applies to all compile time testing, not just
in-repository testing.)

Maybe just require a link to a well-formatted previous results file?

Or maybe you are right, that this is something where the
"check-compile" rule spits out a log file, which is then posted to
gcc-testresults, and then something server-side does the graphing and
comparing once the two URLs that contain results are given...

> (This is also one thing that makes the libstdc++ performance testsuite less
> useful)

Yes, agreed. What's encouraging with your public results is that they
do seem to correlate with my saved logs.

-benjamin


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]