spec2k comparison of gcc 4.1 and 4.2 on AMD K8

Serge Belyshev belyshev@depni.sinp.msu.ru
Sun Feb 25 22:01:00 GMT 2007

"Vladimir N. Makarov" <vmakarov@redhat.com> writes:

> I run SPEC2000 several times per week and always look at 3 runs (to be
> sure that is nothing wrong happened) but I never saw such big
> "confidence" intervals (as I understand that is difference between max
> and min of 3 runs divided by the score). [...]

No, it is much more complex than that, I've used generally accepted
definition of a confidence interval, see http://en.wikipedia.org/wiki/Confidence_interval
which basically tells that with 95% probabilty (the confidence level I've choosed)
true value lies in this interval.

I've used conservative estimate of confidence intervals in this case
because I didn't assume gaussian distribution of numbers which I
reported as difference between two run times, and this estimate is somewhat
bigger than difference between max and min of 3 runs :)

> [...] If the machine has only 512 Mb memory (even they
> write that it is enough for SPEC2000), the scores for some benchmark
> programs may be unstable.  [...]

My box is equipped with 2Gigs of RAM so I believe this is not the case,
Also the computer was *absolutely* idle when it was running spec2k.
(booted with init=/bin/sh and no other processes were running).

And no,
> [...] acknowledge that I never ran SPEC2000 on AMD machines and some
> processors generates less "confident intervals". [...]
this is not the case, I'm absolutely sure.

More information about the Gcc mailing list