Re: Regression testing...

[Joe Buck <jbuck at synopsys dot COM>]

> I would like to see advancement made in terms of the testsuite.  In
> particular, I would like to add back the infrastructure that was
> appropriately donated to the FSF (testsuite/lib and old-deja.exp), and
> then work on a list of known clean testcases.

Actually, we can use probably 95% or more of the current testsuite (maybe
99%) based on an email discussion I had with Eben Moglen a while back
(he's the FSF's lawyer).  We've been remiss on pushing forward the
testsuite sanitization project (I'd forgotten all about it).

The SC's going have to do some work on this, and we should bump up the
priority.  There may be a delay if we have to run the guidelines past
Prof. Moglen (he's a busy man).

> I would like the see written down the guidelines given to the gcc team
> by the FSF for what can be considered ours, so that I may use these
> guidelines as the FSF intended to sort though my part of the testsuite
> for C++ (I know it best), and add to the file that contains the list
> clean testcases the files I think are clean.

If your testcase is an entire program or major routine that is useful on
its own (to do something other than be a test case), or if it contains a
copyright notice, or if it is an identifiable part of a known program,
it's a problem (unless the copyright is held by FSF, or it's under the GPL
or other free software/open source license).  Otherwise we can probably
include it.  The idea is that we don't need to be totally anal-retentive.
We just want to make the likelihood of complaint very small, and have it
be that any rational complainer would be satisfied with us simply removing
the test from the testsuite (that is, any claim that we'd damaged them
by accidentally including their code against their wishes would be
laughable).

> Then, I would like to see the release and snapshot process include the
> clean version of the testsuite.  The unclean portions we can migrate
> to unclean directories.

This seems right.

> I haven't heard much in the way of planning concerning the testsuite,
> and I don't know if this fits in with how things need to be done or
> not, but I thought I might try and get the ball rolling with a
> suggestion.

Thanks for bringing this up.  Now we on the SC have to get off our butts;
we'll then need volunteers to do some reviews.