This is the mail archive of the
mailing list for the GCC project.
Re: clang vs free software
- From: Vladimir Makarov <vmakarov at redhat dot com>
- To: Chris Lattner <clattner at apple dot com>, Steven Bosscher <stevenb dot gcc at gmail dot com>
- Cc: esr at thyrsus dot com, Helmut Eller <eller dot helmut at gmail dot com>, GCC Mailing List <gcc at gcc dot gnu dot org>
- Date: Thu, 23 Jan 2014 23:52:00 -0500
- Subject: Re: clang vs free software
- Authentication-results: sourceware.org; auth=none
- References: <CAJnXXoi2MLpZWxOxknR=mNR91JdZcHrKRsqYZSWY373fvwxObg at mail dot gmail dot com> <87eh439w1n dot fsf at uwakimon dot sk dot tsukuba dot ac dot jp> <CAJnXXojjSAWL8cqZp0X16xa81R73huywtTS90p6O3CpRaPOiDQ at mail dot gmail dot com> <jwvwqhu8zcg dot fsf-monnier+emacs at gnu dot org> <87ha8yqvup dot fsf at engster dot org> <E1W5cXI-0000j4-8x at fencepost dot gnu dot org> <CAJnXXoiuzZhjDGpvXY7psee=+bXn1rB+GdELYP0FS0CuWPqYeQ at mail dot gmail dot com> <E1W6HwP-0001WU-Tg at fencepost dot gnu dot org> <87r47zezcc dot fsf at fencepost dot gnu dot org> <m2eh3ykc3y dot fsf at gmail dot com> <20140123174934 dot GA10933 at thyrsus dot com> <CABu31nMSPdsGU-sAcO_HnFRetqQDgk5fQ3R12aYA0aqTvgdx+A at mail dot gmail dot com> <8E23EFFC-88C3-48BE-97AB-77207DA32499 at apple dot com>
Sorry, I forgot that pdf file is not permitted. Therefore I am
resending my email without it.
On 1/23/2014, 5:56 PM, Chris Lattner wrote:
On Jan 23, 2014, at 12:14 PM, Steven Bosscher <firstname.lastname@example.org> wrote:
(Hint: read http://vmakarov.fedorapeople.org/spec/ as an example of a
better-supported point of view.)
Unrelated to this thread, it would be great for this web page to get updated. You may find it to be "a better-supported point of view", but it is also comparing against clang 3.2, which is from the end of 2012, and a lot has changed since then.
Modern clang versions have its autovectorizor on by default (like GCC's) and generate much better code in general. http://www.phoronix.com/ has done some benchmarks across a wider range of code than just spec (which is notoriously "hacked" by compiler developers) and Clang generates better code (and faster) than GCC in many cases.
I am going to do this when gcc4.9 is released. I have data for gcc4.8
and 3.3 but I never published them. Here is an excerpt from my slides
about the compiler release comparison:
o Intel Haswell 3.4Ghz (i5-4670)
p Common opts: -mtune=corei7 -march=i686
o LLVM: -O3
o GCC: -Ofast -fno-fast-math
o LLVM: -O4
o GCC: -Ofast -fno-fast-math -flto -fwhole-program
o 32-bit: additionally -mpc64
o Only 4 SPECFP2000 (C) tests as LLVM has no Fortran FE
SPECInt LLVM-3.2 LLVM-3.3 GCC-4.7 GCC-4.8
3302 (0.0%) 3305 (+0.06%) 3525 (+6.7%) 3543 (+7.3%)
3592 (0.0%) 3596 (+0.06%) 3704 (+3.1%) 3740 (+4.2%)
3537 (0.0%) 3517 (-0.57%) 3692 (+4.4%) 3724 (+5.4%)
3787 (0.0%) 3789 (+0.05%) 3806 (+0.5%) 3846 (+1.6%)
SPECFP LLVM-3.2 LLVM-3.3 GCC-4.7 GCC-4.8
3583 (0.0%) 3580 (-0.08%) 4888 (+36%) 4985 (+39%)
3661 (0.0%) 3668 (+0.19%) 4878 (+33%) 5011 (+37%)
5468 (0.0%) 5431 (-0.68%) 5799 (+6%) 5917 (+8.2%)
5659 (0.0%) 5659 (0.0%) 5881 (+3.9%) 6005 (+6.1%)
o LLVM 3.2 -> 3.3: No progress, even small 64-bit peak degradation (0.6%)
o GCC4.7-> 4.8: Steady progress (1%-4%)
o Still LLVM is dangerously close to GCC on some tests (1.6% on 64-bit
I also already have some comparison data for the trunk and llvm 3.4 but
I don't want to publish it here as the trunk is not a release.
As for Phoronix, so far I saw several pitfalls in their testing methodology:
o Micro-benchmarking. E.g. favorite benchmark Scimark2 contains a few
tests with only one small hot loop, like LU-factorization where most
benchmark time is spent in 2-lines loop. It means that the worse
results for GCC can be easily fixed as Jakub Jelinek recently improved
Scimark SOR by 42% by a small patch:
o Comparing LLVM and GCC on Fortran benchmarks. LLVM has no fortran FE
and just quietly call system GCC. So comparison of LLVM and GCC on
Fortran benchmarks means comparison of system GCC and a given GCC.
o IMHO, the data in articles lack credability may be because a wrong
setup (by me or by phoronix). E.g. I tried to reproduce Scimark results
for GCC4.8 and LLVM3.3 from his article "LLVM Clang 3.4 Already Has Some
Phoronix used i7-4770K for this. I used the closest machine I found
i5-4670 (with switched turbo mode off). The important difference is
0.1Ghz in frequency (3.5Ghz vs. 3.4 Ghz). I got GCC Scimark (-large)
composite score close to the article when I used -O and still on my
machine the composite score was 20% higher than the article reports
although the article says that -O3 -march=core-avx were used.
o Phoronix articles about LLVM and GCC usually contains a lot of
negative emotions about GCC and positive ones about LLVM. Such bias to
LLVM is suspicious at least for me and make me feel Phoronix as just
LLVM marketing machine.
Still I like that LLVM did a good progress in generated code
performance, it makes GCC people working on optimizations (including me)
to justify the importance of their work. In overall, competition is a
good thing for LLVM and GCC as it stimulates compiler developing in
faster pace. One time it was GCC vs Open64. Next time it would be GCC
and something else or LLVM and something else. Who knows.