Re: detailed regression report for 2.95/i686-pc-sco3.2v5.0.5

[mrs at wrs dot com (Mike Stump)]

> Date: Thu, 3 Jun 1999 22:13:18 -0500
> From: Robert Lipe <robertl@sco.com>

> > If you had used contrib/compare_tests this would have been more
> > readable.  :-)

> Actually, I experimented with that beforehand.

> compare_tests gets so confused by the multiple runs across the
> multilibs that it's easier to pull a unified diff on the output of
> test_summary on the respective runs and annotate them than it is to
> try to backfill the information into compare_tests.

Ah, that problem.  Yes.  I first saw it years ago when Bob (Hi Bob)
made dejagnu do multiple target testing from one invocation of
dejagnu.

The easiest of ways to enhance the bugger without making each testcase
line overly long, is to introduce one new semantic into the dejagnu
output format.  Sat something like:

GROUP: arm-sun-solaris2.5.1 gcc little-indian/PIC
PASS: test1.c

The semantics would be to modify the name of the testcase into:

PASS: arm-sun-solaris2.5.1 gcc little-indian/PIC test1.c

for all testcases up to the next group.  Maybe there should also be an
ENDGROUP: to unset the prefix.  Then the testcase name is again
unique, as it was designed and intended.  Also, at this point, we
might want to have the log and summary file name choosen from the
toplevel makefile, and run all the testing results into _one_ file.
Thus simplifying finding the test results as well.

Or, we if didn't want to do this, we could just add the logic into
deja to track the prefix and output it with the testcase name.  I
think it already has all the information to do this.

Sorry, I didn't notice that complex nature of the results, otherwise,
I would not have suggested compare_tests.  Maybe next time I am
playing in dejagnu I will do this up, and submit it.

Anyway, that is the direction as I see it.