This is the mail archive of the gcc@gcc.gnu.org mailing list for the GCC project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: clang vs free software


Sorry, I forgot that pdf file is not permitted. Therefore I am resending my email without it.

On 1/23/2014, 5:56 PM, Chris Lattner wrote:
On Jan 23, 2014, at 12:14 PM, Steven Bosscher <stevenb.gcc@gmail.com> wrote:
(Hint: read http://vmakarov.fedorapeople.org/spec/ as an example of a
better-supported point of view.)

Unrelated to this thread, it would be great for this web page to get updated.  You may find it to be "a better-supported point of view", but it is also comparing against clang 3.2, which is from the end of 2012, and a lot has changed since then.

Modern clang versions have its autovectorizor on by default (like GCC's) and generate much better code in general.  http://www.phoronix.com/ has done some benchmarks across a wider range of code than just spec (which is notoriously "hacked" by compiler developers) and Clang generates better code (and faster) than GCC in many cases.

-Chris


I am going to do this when gcc4.9 is released. I have data for gcc4.8 and 3.3 but I never published them. Here is an excerpt from my slides about the compiler release comparison:

------------------------------

SPEC2000 Setup

o Intel Haswell 3.4Ghz (i5-4670)
p Common opts: -mtune=corei7 -march=i686
o peak:
  o LLVM: -O3
  o GCC: -Ofast -fno-fast-math
o LTO:
   o LLVM: -O4
   o GCC: -Ofast -fno-fast-math -flto -fwhole-program
o 32-bit: additionally -mpc64
o Only 4 SPECFP2000 (C) tests as LLVM has no Fortran FE


SPEC2000 Results

SPECInt LLVM-3.2    LLVM-3.3      GCC-4.7      GCC-4.8
32-bit peak
        3302 (0.0%) 3305 (+0.06%) 3525 (+6.7%) 3543 (+7.3%)
32-bit LTO
        3592 (0.0%) 3596 (+0.06%) 3704 (+3.1%) 3740 (+4.2%)
64-bit peak
        3537 (0.0%) 3517 (-0.57%) 3692 (+4.4%) 3724 (+5.4%)
64-bit LTO
        3787 (0.0%) 3789 (+0.05%) 3806 (+0.5%) 3846 (+1.6%)

SPECFP  LLVM-3.2    LLVM-3.3      GCC-4.7      GCC-4.8
32-bit peak
        3583 (0.0%) 3580 (-0.08%) 4888 (+36%)  4985 (+39%)
32-bit LTO
        3661 (0.0%) 3668 (+0.19%) 4878 (+33%)  5011 (+37%)
64-bit peak
        5468 (0.0%) 5431 (-0.68%) 5799 (+6%)   5917 (+8.2%)
64-bit LTO
        5659 (0.0%) 5659 (0.0%)   5881 (+3.9%) 6005 (+6.1%)


Conclusions

o LLVM 3.2 -> 3.3: No progress, even small 64-bit peak degradation (0.6%)
o GCC4.7-> 4.8: Steady progress (1%-4%)
o Still LLVM is dangerously close to GCC on some tests (1.6% on 64-bit LTO SPECInt)

----------------------------------------


I also already have some comparison data for the trunk and llvm 3.4 but I don't want to publish it here as the trunk is not a release.

As for Phoronix, so far I saw several pitfalls in their testing methodology:

o Micro-benchmarking. E.g. favorite benchmark Scimark2 contains a few tests with only one small hot loop, like LU-factorization where most benchmark time is spent in 2-lines loop. It means that the worse results for GCC can be easily fixed as Jakub Jelinek recently improved Scimark SOR by 42% by a small patch:

http://gcc.gnu.org/ml/gcc-patches/2013-12/msg01986.html

o Comparing LLVM and GCC on Fortran benchmarks. LLVM has no fortran FE and just quietly call system GCC. So comparison of LLVM and GCC on Fortran benchmarks means comparison of system GCC and a given GCC.

o IMHO, the data in articles lack credability may be because a wrong setup (by me or by phoronix). E.g. I tried to reproduce Scimark results for GCC4.8 and LLVM3.3 from his article "LLVM Clang 3.4 Already Has Some Performance Changes":

http://www.phoronix.com/scan.php?page=article&item=llvm_clang34_first&num=2

Phoronix used i7-4770K for this. I used the closest machine I found i5-4670 (with switched turbo mode off). The important difference is 0.1Ghz in frequency (3.5Ghz vs. 3.4 Ghz). I got GCC Scimark (-large) composite score close to the article when I used -O and still on my machine the composite score was 20% higher than the article reports although the article says that -O3 -march=core-avx were used.

o Phoronix articles about LLVM and GCC usually contains a lot of negative emotions about GCC and positive ones about LLVM. Such bias to LLVM is suspicious at least for me and make me feel Phoronix as just LLVM marketing machine.

Still I like that LLVM did a good progress in generated code performance, it makes GCC people working on optimizations (including me) to justify the importance of their work. In overall, competition is a good thing for LLVM and GCC as it stimulates compiler developing in faster pace. One time it was GCC vs Open64. Next time it would be GCC and something else or LLVM and something else. Who knows.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]