The code of the scripts can be cloned from git://repo.or.cz/gcc-perf-regression-tester.git
This set of scripts are used to spot performance regressions of GCC for various benchmarks:
parsec (free multithreaded benchmark)
pb05 (free benchmark, Fortran)
- polykernels (free benchmark, C - for polyhedral optimizations)
SPEC2006 (commercial benchmark, C, Fortran and C++)
Most of these benchmarks have to be installed in their specific directory. For this, one may use the bmks/*/install.sh scripts for the free benchmarks.
A run of the tester go.sh will produce a performance report for each of the benchmarks that have been installed. A perf report contains:
- a GCC commit: perf report is git tagged
- compiler flags: ofast, lto, graphite, etc.
- execution time of the benchmark
The aim is to make the experiments deterministic and reproducible by storing enough information. Perf reports are stored permanently in read only mode, attached as tags to the tested GCC commits.
The perf analysis script get.sh is run independently in a second stage: it gathers its information from git tags.
The tester infrastructure may use gearman to manage/scatter testing tasks. Gearman makes the infrastructure extensible: can add/drop test machines at anytime.