This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: spec2k comparison of gcc 4.1 and 4.2 on AMD K8
- From: Serge Belyshev <belyshev at depni dot sinp dot msu dot ru>
- To: "Vladimir N. Makarov" <vmakarov at redhat dot com>
- Cc: gcc at gcc dot gnu dot org, Mark Mitchell <mark at codesourcery dot com>
- Date: Mon, 26 Feb 2007 01:01:47 +0300
- Subject: Re: spec2k comparison of gcc 4.1 and 4.2 on AMD K8
- References: <87wt27tr0h.fsf@depni.sinp.msu.ru> <45E1FDAC.9080207@redhat.com>
"Vladimir N. Makarov" <vmakarov@redhat.com> writes:
> I run SPEC2000 several times per week and always look at 3 runs (to be
> sure that is nothing wrong happened) but I never saw such big
> "confidence" intervals (as I understand that is difference between max
> and min of 3 runs divided by the score). [...]
No, it is much more complex than that, I've used generally accepted
definition of a confidence interval, see http://en.wikipedia.org/wiki/Confidence_interval
which basically tells that with 95% probabilty (the confidence level I've choosed)
true value lies in this interval.
I've used conservative estimate of confidence intervals in this case
because I didn't assume gaussian distribution of numbers which I
reported as difference between two run times, and this estimate is somewhat
bigger than difference between max and min of 3 runs :)
> [...] If the machine has only 512 Mb memory (even they
> write that it is enough for SPEC2000), the scores for some benchmark
> programs may be unstable. [...]
My box is equipped with 2Gigs of RAM so I believe this is not the case,
Also the computer was *absolutely* idle when it was running spec2k.
(booted with init=/bin/sh and no other processes were running).
And no,
> [...] acknowledge that I never ran SPEC2000 on AMD machines and some
> processors generates less "confident intervals". [...]
this is not the case, I'm absolutely sure.