Re: LMbench as gcc performance regression test?

[Dan Kegel <dank at kegel dot com>]

Daniel Jacobowitz wrote:
> On Sun, Aug 31, 2003 at 12:21:37AM -0700, Dan Kegel wrote:
>
> > http://cs.nmu.edu/~benchmark/ has an interesting little graph
> > of LMBench results vs. Linux kernel version, all done with the
> > same compiler.
> >
> > Has anyone seen a similar graph showing LMBench results vs. gcc version,
> > all done with the same Linux kernel?
> > And does everyone agree that's a meaningful way to compare the
> > performance of code generated by different compilers?
>
> It's been a while since I looked at lmbench but: why do you think this
> would be useful?  It's a system and kernel benchmark; I doubt
> optimization makes much difference at all.

I need to make sure that moving to a newer compiler for our kernel
will cause no performance regressions.  Before bothering to bring up a
real-world networking application and measuring its performance
under the new compiler, it seems sensible to use a couple microbenchmarks
to verify that identifiable parts of the system have
not degraded in performance.

I myself am quite convinced I need to move to a newer compiler,
since I keep running into problems building various things with
old compilers, but my users are very conservative and skeptical;
I have to build a solid case for updating.  Hence the insane amount
of time I spent figuring out and documenting how to build and test
the various versions of gcc and glibc (http://kegel.com/crosstool),
and then understanding the regression test failures.
- Dan

--
Dan Kegel
http://www.kegel.com
http://counter.li.org/cgi-bin/runscript/display-person.cgi?user=78045