This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: clang vs free software
- From: Jakub Jelinek <jakub at redhat dot com>
- To: Vladimir Makarov <vmakarov at redhat dot com>
- Cc: Chris Lattner <clattner at apple dot com>, Steven Bosscher <stevenb dot gcc at gmail dot com>, esr at thyrsus dot com, Helmut Eller <eller dot helmut at gmail dot com>, GCC Mailing List <gcc at gcc dot gnu dot org>
- Date: Fri, 24 Jan 2014 08:21:45 +0100
- Subject: Re: clang vs free software
- Authentication-results: sourceware.org; auth=none
- References: <87ha8yqvup dot fsf at engster dot org> <E1W5cXI-0000j4-8x at fencepost dot gnu dot org> <CAJnXXoiuzZhjDGpvXY7psee=+bXn1rB+GdELYP0FS0CuWPqYeQ at mail dot gmail dot com> <E1W6HwP-0001WU-Tg at fencepost dot gnu dot org> <87r47zezcc dot fsf at fencepost dot gnu dot org> <m2eh3ykc3y dot fsf at gmail dot com> <20140123174934 dot GA10933 at thyrsus dot com> <CABu31nMSPdsGU-sAcO_HnFRetqQDgk5fQ3R12aYA0aqTvgdx+A at mail dot gmail dot com> <8E23EFFC-88C3-48BE-97AB-77207DA32499 at apple dot com> <52E1F170 dot 603 at redhat dot com>
- Reply-to: Jakub Jelinek <jakub at redhat dot com>
On Thu, Jan 23, 2014 at 11:52:00PM -0500, Vladimir Makarov wrote:
> o IMHO, the data in articles lack credability may be because a wrong
> setup (by me or by phoronix). E.g. I tried to reproduce Scimark
> results for GCC4.8 and LLVM3.3 from his article "LLVM Clang 3.4
> Already Has Some Performance Changes":
>
> http://www.phoronix.com/scan.php?page=article&item=llvm_clang34_first&num=2
>
> Phoronix used i7-4770K for this. I used the closest machine I found
> i5-4670 (with switched turbo mode off). The important difference is
> 0.1Ghz in frequency (3.5Ghz vs. 3.4 Ghz). I got GCC Scimark
> (-large) composite score close to the article when I used -O and
> still on my machine the composite score was 20% higher than the
> article reports although the article says that -O3 -march=core-avx
> were used.
Yeah, that is my experience too, e.g. on
http://www.phoronix.com/scan.php?page=article&item=llvm34_gcc49_compilers&num=3
Phoronix claims the LU benchmark improved 80% with LLVM 3.4 and ahead of GCC,
but I couldn't reproduce anything close to that, while LLVM 3.4 slightly
improved compared to LLVM 3.3, it was still comparable to GCC 4.8 and behind
GCC 4.9. All that matters in the benchmark is a single loop though:
for (jj=j+1; jj<N; jj++)
Aii[jj] -= AiiJ * Aj[jj];
vectorized by all tested compilers, so clearly a microbenchmark.
It is not that long that Phoronix used to compile some of the benchmarks
e.g. with -O0 and claim they were compiled with -O3, or tune for a
completely different CPU than what it has been tested on, their articles often
comment on the numbers in quite biased way without actually bothering
to look at why are the numbers changing, what number changes are within the
noise and if something changes more than that and is repeatable, why it
happened, so their numbers and benchmarks can't be considered the most
credible benchmarks out there.
Jakub