GCC 4.6 performance regressions

[Michael Larabel]

Hi Everyone,

I've been following the thread already from the start (I'm on the list). 
Matthew Tippett (CC'ed now as well) and I have already been working on 
some ways to help further and one of us should have some more 
information to be presented shortly. If any of you have any other 
questions please feel free to let us know.

-- Michael

On 02/09/2011 02:34 AM, Sebastian Pop wrote:
> On Wed, Feb 9, 2011 at 00:20, Tony Poppleton<tony.poppleton@gmail.com>  wrote:
>>> While I appreciate Phoronix as a booster site, their benchmarking
>>> practice often seems very dodgy; I'd take the results with a large grain
>>> of salt....
>> The main reason I posted the link in the first place was because it
>> was reflecting my own emperical evidence for the application I am
>> working on; the march/mtune flags (on various corei* cpus) are
>> actually detrimental to performance (albeit by at most 10% - but
>> still, not the performance boost I was hoping for).
>>
>> I had been trying for the last few weeks to strip down my application
>> into a mini-benchmark I could create a PR out of, however it is
>> tougher than expected and was hoping the Phoronix article was going to
>> offer a quicker route to finding performance regressions than my code
>> - as their coverage was a lot wider.  Anyway, apparently this is not
>> the case, so back to my original work...
>>
>> Would it not be in the best interests for both GCC and Phoronix to
>> rectify the problems in their benchmarks?
> I agree with you here: for many reasons it would be interesting to
> have Phoronix as a free alternative to the SPEC benchmarks.
>
>>   Could we not contact the
>> authors of the article and point out how they can make their
>> benchmarks better?  I would be happy to contact them myself, however I
>> think it would be far more effective (and valid) coming from a GCC
>> maintainer.
> I put Michael in CC.
>
>> Points they have apparently missed so far are;
>>   - clarify which compiler flags they are using
> Also, the mechanism used to change the flags for a benchmark should be
> documented and should produce the intended behaviour.
>
> For example x264 defines CFLAGS="-O4 -ffast-math $CFLAGS", and so
> building this benchmark with CFLAGS="-O2" would have no effect.
>
> Sebastian
>
>>   - don't run make -j when looking at compile times
>>   - ensure they are using --enable-checking=release when benchmarking a snapshot
>>   - in general, as much information as possible as to their setup and
>> usage, to make the results easily repeatable
>>
>> Out of interest, has their been much communication in the past between
>> GCC and Phoronix to address any of these issues in their previous
>> benchmarks?
>>
>> Tony
>>