This is the mail archive of the
mailing list for the GCC project.
Re: GCC-4.5.0 comparison with previous releases and LLVM-2.7 on SPEC2000 for x86/x86_64
- From: Xinliang David Li <davidxl at google dot com>
- To: Vladimir Makarov <vmakarov at redhat dot com>
- Cc: "gcc.gcc.gnu.org" <gcc at gcc dot gnu dot org>
- Date: Thu, 29 Apr 2010 11:17:22 -0700
- Subject: Re: GCC-4.5.0 comparison with previous releases and LLVM-2.7 on SPEC2000 for x86/x86_64
- References: <4BD9B2EB.email@example.com>
On Thu, Apr 29, 2010 at 9:25 AM, Vladimir Makarov <firstname.lastname@example.org> wrote:
> ?GCC-4.5.0 and LLVM-2.7 were released recently. ?To understand
> where we stand after releasing GCC-4.5.0 I benchmarked it on SPEC2000
> for x86/x86-64 and posted the comparison of it with the
> previous GCC releases and LLVM-2.7.
> ?Even benchmarking SPEC2000 takes a lot of time on the fastest
> machine I have. So I don't plan to use SPEC2006 for this in near
> ?You can find the comparison on
> http://vmakarov.fedorapeople.org/spec/ (please just click links at the
> bottom of the left frame starting with link "GCC release comparison").
> ?If you need exact numbers, please use the tables (the links to them
> are also given) which were used to generate the corresponding bar
> ?In general GCC-4.5.0 became faster (upto 10%) in -O2 mode. ?This is
> first considerable compilation speed improvement since GCC-4.2.
> GCC-4.5.0 generates a better (1-2% in average upto 4% for x86-64
> SPECFP2000 in -O2 mode) code too in comparison with the previous
> release. ?That is not including LTO and Graphite which can gives even
> more (especially LTO) in many cases.
> ?GCC-4.5.0 has new big optimizations LTO and Graphite (more
> accurately graphite was introduced in the previous release).
> Therefore I ran additional benchmarks to test them.
> ?LTO is a promising technology especially for integer benchmarks for
> which it results in smaller and faster code. ?But it might result in
> degradations too on SPECFP2000 mainly because of big degradations on a
> few benchmarks like wupwise or facerec. ?Another annoying thing about
> LTO, it considerably slows down the compiler.
The LTO improvement on spec2000int is is only 1.86%
4.5 4.5+lto Improvement
164.gzip 955 950 -0.52% <-- degrade
175.vpr 588 594 1.02%
176.gcc 1211 1216 0.41%
181.mcf 699 698 -0.14%
186.crafty 1011 987 -2.37% <--- degrade
197.parser 792 813 2.65%
252.eon 1026 1023 -0.29% <-- degrade
253.perlbmk 1312 1294 -1.37% <-- degrade
254.gap 1021 1037 1.57%
255.vortex 1123 1319 17.45%
256.bzip2 737 768 4.21%
300.twolf 773 779 0.78%
SPECint2000 913 930 1.86%
This matches our previous observation that to bring the best out of
LTO, FDO is also needed. (As a reference, LIPO improves over plain FDO
by ~4.5%, vortex improves 23%). You will probably see even smaller
improvement in SPEC2006.
It would be great if there is number collected comparing LTO + FDO vs
plain FDO in the same setup.
> ?Currently Graphite gives small improvements on x86 (one exception is
> 2% for peak x86 SPECFP2000) and mostly degradation on x86_64 (with
> maximum one more than 10% for SPECFP2000 because of big degradations
> on mgrid and swim). ?So further work is needed on the project because
> it seems not mature yet.
> ?As for LLVM, LLVM became slower (e.g. in comparison with llvm-2.5 on
> 15%-50% for x86-64). ?So the gap between compilation speed of GCC and
> LLVM decreased and sometimes achieves 4% on x86_64 and 8% on x86 (both
> for SPECInt2000 in -O2 mode). ?May be I am wrong but I don't think
> CLANG will improve this situation significantly (in -O2 and -O3 mode)
> because optimizations still take most of time of any serious
> optimizing compiler.
> ?LLVM did a progress in code performance especially for floating
> point benchmarks. ?But the gap between LLVM-2.7 and GCC-4.5 in peak
> performance (not including GCC LTO and Graphite) still 6-7% on
> SPECInt200 and 13-17% on SPECFP2000.
> ?In general, IMHO GCC-4.5.0 is a good and promising release.