Re: [try QMTest]

[Loren James Rittle <rittle at latour dot rsch dot comm dot mot dot com>]

Mark asked me to publicly report this finding: I have installed
QMTest-1.1.5 on i386-unknown-freebsd4.6.  It works with my local
python-2.2 installation.  I can also report that the native -j1 target
results look reasonable compared to current dejagnu results.

[Mark, new information:] However, I see many more failures with, e.g.:

  gmake QMTESTRUNFLAGS\='-j 2 --seed 1' qmtest-g++

than, e.g.:

  gmake QMTESTRUNFLAGS\='-j 1 --seed 1' qmtest-g++

First of all, I see that the invoked command lines are sometimes
different.  E.g. gpp.dg.inherit.template-as-base is missing all -I
switches in -j2 case.  gpp.old-deja.jason.ref12 appears to have the
same command line in both cases, yet CompilerTest.execution_exit_code
is 1 with no other useful information in the (qmr) log file.  Do you
see it, or will you need a more complete report?

[Mark, early *opinion* after studying your QMTest PDF manual:] In the
abstract, I see intriguing benefits to this change: (1) an
architecture to properly support fine-grain parallel execution of test
cases (as I will soon have access to a 42-CPU cluster, I can dig it);
(2) use of "structured" XML log files to remove introduction of errors
during parsing for later analysis and regression checking.

However, dejagnu has grown big and nasty in some corners for a reason.
I suppose that your (Mark's) position is that much of the extra
marking information about what test cases don't work on various
platforms should go into baseline qmr files (checked by a platform
expert) instead of being hard-coded in either dejagnu itself and/or
the gcc testsuite.  If so, I buy that it could work based on my
current understanding.

I have no opinion on the (current?) loss of cross-checking features of
dejagnu since I don't use that.  BTW, my knowledge of TCL is minimal;
of Python about zero before I looked at QMTest source code.

Regards,
Loren