This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
[BENCHMARK] Statistical representation of IO load results with contest
- From: Con Kolivas <conman at kolivas dot net>
- To: linux-kernel at vger dot kernel dot org
- Cc: Andrew Morton <akpm at digeo dot com>,Ryan Anderson <ryan at michonline dot com>, Ingo Molnar <mingo at elte dot hu>,"Richard B. Johnson" <root at chaos dot analogic dot com>,Oliver Xymoron <oxymoron at waste dot org>,Daniel Jacobowitz <drow at false dot org>,Paolo Ciarrocchi <ciarrocchi at linuxmail dot org>,Robinson Maureira Castillo <rmaureira at alumno dot inacap dot cl>,gcc at gcc dot gnu dot org
- Date: Tue, 24 Sep 2002 09:26:38 +1000
- Subject: [BENCHMARK] Statistical representation of IO load results with contest
Thank you all who responded with suggestions on how to get useful data out of
the IO load module from contest. These are _new_ results with a
sync,swapoff,swapon before conducting just the IO load. I have digested all your
suggestions and come up with the following:
n=5 for number of samples
Kernel Mean CI(95%)
2.5.38 411 344-477
2.5.39-gcc32 371 224-519
2.5.38-mm2 95 84-105
The mean is a simple average of the results, and the CI(95%) are the 95%
confidence intervals the mean lies between those numbers. These numbers seem to
be the most useful for comparison.
Comparing 2.5.38(gcc2.95.3) with 2.5.38(gcc3.2) there is NO significant
difference (p 0.56)
Comparing 2.5.38 with 2.5.38-mm2 there is a significant diffence (p<0.001)
After playing with all these it appears I should do the following to contest:
Add sync,swapoff,swapon before each load
Perform noload and process_load twice to ensure no abnormal results
Perform mem_load 3 times
Perform IO_fullmem 5 times (and rename it just IO_load)
Drop IO_halfmem (adds no more useful information and just adds time).
Do a statistical analysis like the above when posting information.
Comments?
Con