This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: Performance regression testing?
On Tue, 29 Nov 2005, Mike Stump wrote:
> > What field order looks better to you? I'm agnostic, except I'd
> > like to keep one and the same field delimiter except for the
> > result, and it's *slightly* easier to keep it as "," (as in the
> > original csibe output).
>
> 4.1-sparc-r104567/my-perf-suite.sum
> mainline-sparc-r102355/my-perf-suite.sum
>
> name_of_analysis_prog 4.1-sparc-r104567/my-perf-suite.sum mainline-
> sparc-r102355/my-perf-suite.sum
>
> exactly like dejagnu. Here, there is no previous, there is only the
> first argument to the analysis program, which, by definition names
> the `previous' run.
No, I mean the order of the fields of results for sub-tests
*within* the result file, as you say you wanted some other. It
seems we're talking past each other; I can't parse your reply as
a reply, and I don't really understand how your
name_of_analysis_prog runs would fit in the dejagnu framework.
Hey, if you don't want to read the csibe.exp source, you'll have
to wait until there's something I think may be worthy of a
presentation rather than just a dump of w.i.p. :-)
Oh well: in what I play with currently, baselines are input in
multiple arbitrarily named files: each may contain one (part of)
or more arbitrarily named (except for the delimiter) baselines.
As baseline output, there will be one result file (TBD),
containing for convenience as input to any other run, lines
marked "previous" (for the current result) and "best" (an
updated or created baseline named "best"). There are lots of
lines of data in a baseline (exactly what is TBD, see csibe.exp
for example). This file may be edited, split up in multiple
files or whatever, serving as baseline input to a future run.
If the raw CSiBE output data is needed for e.g. graphs, it's
available too. Baseline data is just trivially derived from it.
The dejagnu output is simply comparisons of each baseline to the
current results (criterias TBD, example given in code). I'm not
sure an external test-program should be used by the testsuite,
if all it does is a "<=" comparison. Maybe as an option,
default off? For post-testsuite-analysis, it's certainly simple
enough to do what you mention.
By the way, I don't want baseline data having .sum suffix since
it doesn't have dejagnu PASS/FAIL contents. For one, the
contrib/test_summary script wouldn't like it.
brgds, H-P