Re: QMTest and the G++ testsuite

[Zack Weinberg <zack at codesourcery dot com>]

On Wed, May 22, 2002 at 10:17:24PM -0700, Mark Mitchell wrote:
> 
> 
> --On Wednesday, May 22, 2002 09:18:27 PM -0300 Alexandre Oliva 
> <aoliva@redhat.com> wrote:
> 
> >On May 22, 2002, Mark Mitchell <mark@codesourcery.com> wrote:
> >
> >>My goal is to get the process started -- I've offerred to do what it
> >>takes for GCC.
> >
> >My concern is that, by going down that path, we may soon get into a
> >situation in which most people would trade DejaGNU for QMTest because
> >of the features it offers (and you're doing a good job at showing me
> >and others that it does have some significant advantages), in such a
> >way that we may soon start getting chunks of the testsuite that depend
> >exclusively on QMTest, and then there will be no way back.
> 
> I think we should try to prevent that.  It would be unwise to commit
> ourselves until we're sure we want to go whole hog.  That's one of the
> nice things about this approach -- we don't actually have to change
> the tests themselves.

However, I'd like to point out that there are some categories of test
that would be difficult or impossible to express within DejaGNU, but
can be done pretty easily in QMTest.

The example that comes to mind is the "nasty file names" test, which
generates a series of input files whose names (collectively) cover all
the characters that the operating system allows in file names, and
makes sure that the compiler can handle them.  You can't check all
those files into CVS, the set of acceptable names varies with the
operating system.  I tried and failed to do this in DejaGNU.  Granted,
this test requires writing custom Tcl, and would require writing
custom Python in QMTest.

zw