New GCC releases comparison and comparison of GCC4.4 and LLVM2.5 on SPEC2000

[Vladimir Makarov]

Chris Lattner wrote:
>
> On May 12, 2009, at 6:56 AM, Vladimir Makarov wrote:
>
>> A few people asked me to do a new comparison of GCC releases and LLVM 
>> as the new GCC release and LLVM were out recently.
>>
>> You can find the comparison on http://vmakarov.fedorapeople.org/spec/
>>
>> The comparison for x86 (32-bit mode) was done on Pentium4 and  for 
>> x86_64 (64-bit mode) on Core I7.
>>
>> Some changes in the performance were big since GCC 3.2 and it is 
>> sometimes hard to see small changes on the posted graphs.   Therefore 
>> I put original tables used to generate the graphs.
>
> Looking at the llvm 2.5 vs gcc 4.4 comparison is very interesting, 
> thank you for putting this together Vladimir!  I find these numbers 
> particularly interesting because you're comparing simple options like 
> -O2 and -O3 instead of the crazy spec tuning mix :).  This is much 
> more likely to be representative of what real users will get on their 
> apps.
>
> Some random thoughts:
>
> 1. I have a hard time understanding the code size numbers.  Does 10% 
> mean that GCC is generating 10% bigger or 10% smaller code than llvm?
>
The change is reported relative to LLVM.  So 10% means that GCC 
generates 10% bigger code than LLVM and -10% means that GCC generates 
10% less code.
> 2. You change two variables in your configurations: micro architecture 
> and pointer size.  Would you be willing to run x86-32 Core i7 numbers 
> as well?  LLVM in particular is completely untuned for the (really old 
> and quirky) "netburst" architecture, but I'm interested to see how it 
> runs for you on more modern Core i7 or Core2 processors in 32-bit mode.
>
I used the same processor (P4) and options for x86 as for the GCC 
release comparison.  I did not know that LLVM is badly tuned for P4, 
sorry.  I could do the same comparison for x86 on Core i7 without 
specific tuning (there is no tuning for i7 yet) but it takes a lot of 
time.  May be it will be ready on next week.
> 3. Your SPEC FP benchmarks tell me two things: GCC 4.4's fortran 
> support is dramatically better than 4.2's (which llvm 2.5 uses), and 
> your art/mgrid hacks apparently do great stuff :).
>
> 4. Your SPEC INT numbers are more interesting to me.  It looks like 
> you guys have some significant wins in 175.vpr, 197.crafty, and other 
> benchmarks.  At some point, I'll have to see what you guys are doing :)
>
> Thanks for the info, great stuff!
>
> -Chris
>