Re: compiler performance

[Felix Lee <felix dot 1 at canids dot net>]

"Danny Angelo" <danny@malisoft.com.br>:
> Hi, anybody at GCC can study and say anything about this article?
> http://www.osnews.com/story.php?news_id=5602&page=3

note, I don't speak for GCC.

the biggest flaw in his benchmarks is he does

        startTime = clock();
        // ... loop
        stopTime = clock();

since most of his loops don't do any function calls, there's no
particular reason the first call to clock() has to be executed
before the loop.  in C, an optimizer would be allowed to move
some or all of the loop to somewhere before the first clock()
call, so it's not clear to me that he's actually measuring what
he thinks he is.

also, his trig loop looks like this:

        while (i < trigMax)
        {
                sine = sin(i);
                cosine = cos(i);
                tangent = tan(i);
                logarithm = log10(i);
                squareRoot = sqrt(i);
                i++;
        }

an optimizer might know that these functions don't have any
side-effects, so most of the computations can be eliminated, and
then the whole loop can be eliminated.  I'm guessing the
abnormally fast timings he got from the Microsoft compilers is
because they eliminated most of that loop, but it could easily be
something else I don't know about.

in general, I'd want to look at the compiler output for any new
benchmark, to make sure it's measuring what I think it is, but
the author seems unwilling or unable to do that.

he also seems to be generally unaware of language implementation
and optimization issues.  like, he says that on his first
attempt, he got bafflingly fast timings until he changed the
benchmarks to print out a value from the loop.

with this black-box approach to benchmarking, it's going to take
several more iterations before he gets it right.
--