Benchmarks

[Joe Buck]

On Wed, Jun 18, 2003 at 12:08:35PM -0400, Scott Robert Ladd wrote:
> I reject the SPEC benchmarks because of their closed-source nature. In 
> the case of benchmarks (or science, for that matter), closed-source is 
> unacceptable because there is no indepedent verification of methods.

While I agree that the SPEC license is a problem, most of the SPEC tests
themselves are derived from open-source code, and there no problem with
independent verification of SPEC scores (because access to SPEC is
widespread and the reporting standards require that enough information is
provided to duplicate the measurement).

So, if you wrote "only SPEC licensees can independently verify the
results" you would have been more accurate.  Certainly it's better for GCC
if *anyone* who has access to the right hardware can do the verification.