Benchmarking theory

[Joseph S. Myers]

Benchmark results seem to get posted to the gcc list as single figures for
a test and old and new compilers, with assertions that results seem
significant or are consistent between runs.  Why are benchmarks done on
this basis rather than using actual statistical significance tests?  
Could someone point me to appropriate references on the theory of
benchmarking that explain this?

-- 
Joseph S. Myers
jsm28@cam.ac.uk