This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: [BENCHMARK] Corrected gcc3.2 v gcc2.95.3 contest results
- From: Ingo Molnar <mingo at elte dot hu>
- To: Con Kolivas <conman at kolivas dot net>
- Cc: linux-kernel at vger dot kernel dot org, Daniel Jacobowitz <dan at debian dot org>,Andrew Morton <akpm at digeo dot com>, Ville Herva <vherva at niksula dot hut dot fi>,Alexei Podtelezhnikov <apodtele at mccammon dot ucsd dot edu>,<gcc at gcc dot gnu dot org>, Robert Love <rml at tech9 dot net>
- Date: Mon, 23 Sep 2002 09:49:53 +0200 (CEST)
- Subject: Re: [BENCHMARK] Corrected gcc3.2 v gcc2.95.3 contest results
- Reply-to: Ingo Molnar <mingo at elte dot hu>
On Mon, 23 Sep 2002, Con Kolivas wrote:
> IO Full Load:
> 2.5.38 170.21 42%
> 2.5.38-gcc32 230.77 30%
> This time only the IO loads showed a statistically significant
> difference.
how many times are you running each test? You should run them at least
twice (ideally 3 times at least), to establish some sort of statistical
noise measure. Especially IO benchmarks tend to fluctuate very heavily
depending on various things - they are also very dependent on the initial
state - ie. how the pagecache happens to lay out, etc. Ie. a meaningful
measurement result would be something like:
IO Full Load:
2.5.38 170.21 +- 55.21 sec 42%
2.5.38-gcc32 230.77 +- 60.22 sec 30%
where the first column is the average of two measurements, the second
column is the delta of the two measurements divided by 2. This way we can
see the 'spread' of the results.
I simply cannot believe that gcc32 can produce any visible effect in any
of the IO benchmarks, the only explanation would be heavy fluctuation of
IO results.
Ingo