r193277 - in /trunk/contrib: ChangeLog compare_...

[crowl@gcc.gnu.org]

Author: crowl
Date: Wed Nov  7 00:36:04 2012
New Revision: 193277

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=193277
Log:
Add a contrib script for comparing the performance of two sets of
compiler runs.

Usage documentation is in the script.

The script produces output of the form:

$ compare_two_ftime_report_sets "Log0/*perf" "Log3/*perf" 

Arithmetic sample for timevar log files
"Log0/*perf"
and selecting lines containing "TOTAL" with desired confidence 95 is 
trial count is 4, mean is 443.022 (95% confidence in 440.234 to 445.811),
std.deviation is 1.75264, std.error is 0.876322

Arithmetic sample for timevar log files
"Log3/*perf"
and selecting lines containing "TOTAL" with desired confidence 95 is 
trial count is 4, mean is 441.302 (95% confidence in 436.671 to 445.934),
std.deviation is 2.91098, std.error is 1.45549

The first sample appears to be 0.39% larger,
with 60% confidence of being larger.
To reach 95% confidence, you need roughly 14 trials,
assuming the standard deviation is stable, which is iffy.

Tested on x86_64 builds.


Index: contrib/ChangeLog

2012-11-05  Lawrence Crowl  <crowl@google.com>

	* compare_two_ftime_report_sets: New.


Added:
    trunk/contrib/compare_two_ftime_report_sets   (with props)
Modified:
    trunk/contrib/ChangeLog

Propchange: trunk/contrib/compare_two_ftime_report_sets
            ('svn:executable' added)