[PATCH] Tiny predcom improvement (PR tree-optimization/59643)

Vladimir Makarov vmakarov@redhat.com
Wed Jan 1 20:21:00 GMT 2014


On 1/1/2014, 12:50 PM, Jakub Jelinek wrote:
> On Tue, Dec 31, 2013 at 03:22:07PM -0500, Vladimir Makarov wrote:
>> Scimark2 is always used by Phoronix to show how bad GCC in
>> comparison with LLVM.  It is understandable. For some reasons
>> phoronix is very biased to LLVM and, I'd say, a marketing machine
>> for LLVM.
>>
>> They use very small selected benchmarks.  Benchmarking is evil but I
>> believe more SPEC2000/SPEC2006 which use bigger real programs.  With
>> their approach, they could use whetstone instead of Scimark but the
>> results would be very bad for LLVM (e.g. 64-bit code for -O3 on
>> Haswell):
>
> I'm aware of all that, that said putting some minimal effort to increase
> obstackles for their marketing is worth it.
>

I believe that Michael Larabel has no idea what he is measuring.

I don't believe in his data too.  Recently I tried to reproduce Scimark 
results for GCC4.8 and LLVM3.3 from his article "LLVM Clang 3.4 Already 
Has Some Performance Changes":

http://www.phoronix.com/scan.php?page=article&item=llvm_clang34_first&num=2

He used i7-4770K for this.  I used the closest machine I found i5-4670 
(with switched turbo mode off).  The important difference is 0.1Ghz in 
frequency (3.5Ghz vs. 3.4 Ghz).  I got close GCC Scimark (-large) 
composite score when I used -O and still on my machine the composite 
score was 20% higher than the article reports although the article says 
that -O3 -march=core-avx were used.

His articles about LLVM and GCC usually contains a lot of negative 
emotions about GCC and positive ones about LLVM.  I don't know may be he 
tried to communicate with GCC community and got a frustrated experience. 
  That is only my explanation now why he is so biased towards LLVM.

So far I saw several pitfalls in his testing methodology:

o Micro-benchmarking.  You just gave an example LU-factorization in 
Scimark2 where most benchmark time is spent in 2-lines loop.

o Comparing LLVM and GCC on fortran benchmarks.  LLVM has no fortran FE 
and just quietly call system GCC.  So comparison LLVM and GCC on fortran 
benchmarks means comparison system GCC and a given GCC.

o IMO, the data in articles lack credability may be because a wrong setup.



More information about the Gcc-patches mailing list