Re: Hunting for performance regressions in libstdc++

[Dan Kegel <dank at kegel dot com>]

Paolo Carlini wrote:
> First, the "STL" delivered in gcc-3.4.x is *very* similar to the
> original SGI STL, which constitutes our traditional code base: I would
> be *very* surprised to see important performance regressions due to
> those very minor bug fixes. Maybe you are mistaking the whole C++
> runtime for the "STL"?

Probably.

> All the benchmarks that you mentioned make *very* little or no use of
> the C++ library.

That's why I asked; I'm having trouble finding benchmarks.

> Besides a few problems in very specific areas (see Bugzilla, of course)
> we are not aware of performance regressions wrt v2 (quite to the
> contrary!). Don't you believe that a good amount of profiling of your
> application is in order before chasing, generically, "performance
> regressions in libstdc++"? I don't see it mentioned and, in general,
> it's definitely the first thing to do and the way we took care ourselves
> of problems in the past.

Yes, but I'm looking for small things to sic an intern on
before I inflict my real app on him.

Jonathan Wakely wrote:

> http://www.stepanovpapers.com/container_benchmark.cpp
>
> (a modified version of which is included in libstdc++'s performance
> testsuite, IIRC)
>
> You might also try the Stepanov abstraction penalty tests.

Thanks for the suggestions.

Chris Jefferson wrote:

> If you want to find regressions within libstdc++, and compared to 
> SGI's STL, then has been discussed I'd expect 
> a) you won't find alot (except possibly with relation to 
> the allocator, which has changed from time to time), and 

We've noticed that allocators impact our performance, but I
haven't looked at it hard yet.

> b) you'd have to use the same compiler and different libraries.

Yep.  That's my plan.

> A more likely possibility is that the compiler has regressed 
> with respect to compiling libstdc++-v3 (for example, while
> partly fixed now, PR19078 effects std::find, which used used 
> internally by a number of algorithms).

Thanks for the pointer to that PR.

> Finding such regressions could of course also be handy.

Yep.  My *other* intern is looking for compiler performance regressions.

> The obvious question about your app is have you tried profiling it? 
> Unless you are seeing functions from inside the STL high up the 
> list of most-used functions, then it probably isn't their fault :)

Yes, I have, but I haven't found any smoking guns yet.
gcc-3.4.3 is 7% slower than gcc-2.95.3 + SGI STL + local fixes
on this app, but I'm having a hard time tracking it down to any one area,
even using tools like oprofile.
I'm throwing a couple interns at performance issues
in general for a bit before I have them focus on this app,
on the theory that it's better to get them up to speed
on small, knowable problems before giving them something
hard and discouraging.

(This reminds me of the story about the guy who was looking
under a streetlamp for something he'd dropped.  A passerby
asked him where he dropped it.  The guy answered "Over on the
next block".  The passerby asked him "So why are you looking here?",
and the guy answered "Because the light's better here.")
- Dan

--
Trying to get a job as a c++ developer?  See http://kegel.com/academy/getting-hired.html