Re: GCC-4.5.0 comparison with previous releases and LLVM-2.7 on SPEC2000 for x86/x86_64

[Xinliang David Li <davidxl at google dot com>]

I re-measured the performance difference using trunk gcc and trunk
clang/llvm on a core-2 box.  -fno-strict-aliasing is added to gcc
because clang/llvm's type based aliasing is not incomplete and not
enabled by default. I also added -fomit-frame-pointer to clang/llvm as
this is gcc's default. The base option is -O2.

32bit:

            164.gzip                1210                1239      2.44%
             175.vpr                1662                1621     -2.42%
             181.mcf                2733                3109     13.75%
          186.crafty                1812                1721     -5.00%
          197.parser                1328                1289     -2.92%
         253.perlbmk                2086                2580     23.67%
             254.gap                1968                1912     -2.86%
          255.vortex                1842                1965      6.66%
           256.bzip2                1440                1553      7.82%
           300.twolf                2284                2213     -3.08%


64bit:
            164.gzip                1268                1320      4.15%
             175.vpr                1605                1534     -4.42%
             176.gcc                2203                2315      5.08%
             181.mcf                1625                1737      6.85%
          186.crafty                2411                2307     -4.30%
          197.parser                1173                1166     -0.57%
             252.eon                2245                2464      9.72%
         253.perlbmk                2214                2444     10.37%
             254.gap                1987                1978     -0.47%
          255.vortex                2497                2422     -3.00%
           256.bzip2                1585                1740      9.80%
           300.twolf                2294                2281     -0.58%


Though gcc leads LLVM in performance overrall, there are a couple of
benchmarks gcc is worse: vpr and crafty (64bit and 32bit), parser and
twolf (32bit), vortex (64bit).  This needs to be triaged.   gcc
miscompiles gcc and eon in 32bit -- is there a bug tracking the
problem?

Thanks,

David


On Thu, Apr 29, 2010 at 9:25 AM, Vladimir Makarov <vmakarov@redhat.com> wrote:
> ?GCC-4.5.0 and LLVM-2.7 were released recently. ?To understand
> where we stand after releasing GCC-4.5.0 I benchmarked it on SPEC2000
> for x86/x86-64 and posted the comparison of it with the
> previous GCC releases and LLVM-2.7.
>
> ?Even benchmarking SPEC2000 takes a lot of time on the fastest
> machine I have. So I don't plan to use SPEC2006 for this in near
> future.
>
> ?You can find the comparison on
> http://vmakarov.fedorapeople.org/spec/ (please just click links at the
> bottom of the left frame starting with link "GCC release comparison").
>
> ?If you need exact numbers, please use the tables (the links to them
> are also given) which were used to generate the corresponding bar
> graphs.
>
>
> ?In general GCC-4.5.0 became faster (upto 10%) in -O2 mode. ?This is
> first considerable compilation speed improvement since GCC-4.2.
> GCC-4.5.0 generates a better (1-2% in average upto 4% for x86-64
> SPECFP2000 in -O2 mode) code too in comparison with the previous
> release. ?That is not including LTO and Graphite which can gives even
> more (especially LTO) in many cases.
>
> ?GCC-4.5.0 has new big optimizations LTO and Graphite (more
> accurately graphite was introduced in the previous release).
> Therefore I ran additional benchmarks to test them.
>
> ?LTO is a promising technology especially for integer benchmarks for
> which it results in smaller and faster code. ?But it might result in
> degradations too on SPECFP2000 mainly because of big degradations on a
> few benchmarks like wupwise or facerec. ?Another annoying thing about
> LTO, it considerably slows down the compiler.
>
> ?Currently Graphite gives small improvements on x86 (one exception is
> 2% for peak x86 SPECFP2000) and mostly degradation on x86_64 (with
> maximum one more than 10% for SPECFP2000 because of big degradations
> on mgrid and swim). ?So further work is needed on the project because
> it seems not mature yet.
>
> ?As for LLVM, LLVM became slower (e.g. in comparison with llvm-2.5 on
> 15%-50% for x86-64). ?So the gap between compilation speed of GCC and
> LLVM decreased and sometimes achieves 4% on x86_64 and 8% on x86 (both
> for SPECInt2000 in -O2 mode). ?May be I am wrong but I don't think
> CLANG will improve this situation significantly (in -O2 and -O3 mode)
> because optimizations still take most of time of any serious
> optimizing compiler.
>
> ?LLVM did a progress in code performance especially for floating
> point benchmarks. ?But the gap between LLVM-2.7 and GCC-4.5 in peak
> performance (not including GCC LTO and Graphite) still 6-7% on
> SPECInt200 and 13-17% on SPECFP2000.
>
> ?In general, IMHO GCC-4.5.0 is a good and promising release.
>
>