New GCC releases comparison and comparison of GCC4.4 and LLVM2.5 on SPEC2000

[Chris Lattner]

On May 12, 2009, at 6:56 AM, Vladimir Makarov wrote:

> A few people asked me to do a new comparison of GCC releases and  
> LLVM as the new GCC release and LLVM were out recently.
>
> You can find the comparison on http://vmakarov.fedorapeople.org/spec/
>
> The comparison for x86 (32-bit mode) was done on Pentium4 and  for  
> x86_64 (64-bit mode) on Core I7.
>
> Some changes in the performance were big since GCC 3.2 and it is  
> sometimes hard to see small changes on the posted graphs.    
> Therefore I put original tables used to generate the graphs.

Looking at the llvm 2.5 vs gcc 4.4 comparison is very interesting,  
thank you for putting this together Vladimir!  I find these numbers  
particularly interesting because you're comparing simple options like - 
O2 and -O3 instead of the crazy spec tuning mix :).  This is much more  
likely to be representative of what real users will get on their apps.

Some random thoughts:

1. I have a hard time understanding the code size numbers.  Does 10%  
mean that GCC is generating 10% bigger or 10% smaller code than llvm?

2. You change two variables in your configurations: micro architecture  
and pointer size.  Would you be willing to run x86-32 Core i7 numbers  
as well?  LLVM in particular is completely untuned for the (really old  
and quirky) "netburst" architecture, but I'm interested to see how it  
runs for you on more modern Core i7 or Core2 processors in 32-bit mode.

3. Your SPEC FP benchmarks tell me two things: GCC 4.4's fortran  
support is dramatically better than 4.2's (which llvm 2.5 uses), and  
your art/mgrid hacks apparently do great stuff :).

4. Your SPEC INT numbers are more interesting to me.  It looks like  
you guys have some significant wins in 175.vpr, 197.crafty, and other  
benchmarks.  At some point, I'll have to see what you guys are doing :)

Thanks for the info, great stuff!

-Chris