RFC: Automated NIST regression testing

[Jerry DeLisle]

Hi all,

The NIST testsuite contains tests that are automatically checked for pass/fail 
and a number of tests that require inspection.  The tests that require 
inspection are that way because there are multiple possible right answers.

I am considering automating this process by checking the results of the 
pass/fail by simply searching the output.  On the remaining tests that require 
inspection I am proposing creating an output file 'standard' using the test 
suite itself one time and then doing a file compare for those tests each time 
the suite is run from that time forward.

For the test platform I have an older machine (i586, 333 Mhz) here that could be 
used, but I do not usually leave things turned on 24/7 here at home.

Is there a preferred resource available we should run automatic regression 
testing on?

How often should the tests be run?  Daily?

In response to a failure should the script email this list?

With this testing in place I think we can close PR5900.

Any suggestions/comments are welcome.

Best Regards,

Jerry