Re: Automated regression testing

[John Wehle <john at feith dot com>]

> Finally, note that the whole point of the regression tester is that it
> does its very best to only send mail to the person who actually caused
> the regression.

I agree that it's best if the nag message is limited to the person who
actually caused the problem and if it's limited to only one message.
However, it'd be nice if more platforms were covered and that might
involve some tradeoffs.

I actually started to play with this in the interest of being able to
do precommit testing of my own patches on more platforms in a less error
prone way.  It'd be nice if the regression tester would tag the most
recent code which is known to build and pass make check on X platforms
in addition to automatically checking in the expected results.  I'd
then like to run a script which checks out a local copy of the code,
applies my patch, runs a local copy of the tester, and reports to me
the results.

I think that developers are happy to do more precommit testing if it
doesn't involve too much overhead on their part.

-- John
-------------------------------------------------------------------------
|   Feith Systems  |   Voice: 1-215-646-8000  |  Email: john@feith.com  |
|    John Wehle    |     Fax: 1-215-540-5495  |                         |
-------------------------------------------------------------------------