Re: Benchmarking (was Re: [RFC] Moving apart SGI extensions)

[Benjamin Kosnik <bkoz at redhat dot com>]

This is an item that many people want. I suspect that if libstdc++ comes 
up with a solution that works, we could convince g++ to also start using 
this framework. Many of the issues with g++ and inlining could have been 
avoided, or at least known in advance, if this was in place.

> The closest I got to doing any sort of benchmarking for libstdc++-v3 was
> hacking around with mkcheck.in, and the timing code wasn't even mine,
> I think.  :-)

So modest.

> My initial thoughts were to have a Stopwatch class in testsuite_hooks.h.
> Such a class (whatever its name) is a common extension in many commercial C++
> libraries, especially when platform-specific timing routines are possible.
> 
>     Stopwatch   ticker;
>     ....
>     ticker.start();
>     a_string_instance.push_back (half a billion characters);
>     ticker.stop();
>     double clock_ticks = ticker.elapsed();
> 
> etc, etc.

I am in favor of this placement. The memory-limiting code is also here, 
and is conceptually similar (or at least related), I think. 

Actually, doing this like the memory-limiting code is probably the right 
way to proceed. So, the process could be:

1) check at config-time for functionality (GLIBCPP_CONFIGURE_TESTSUITE)
2) do both generic and enabled functionalit in testsuite/testsuite_hooks.h

The design of this functionality is at the discretion of the person doing 
the work, as far as I'm concerned. 

> But I'm sure he's constantly tweaking the tests, so that old copy probably
> wouldn't help us much anyhow.

The Stepanov timer, or the bench++ timer, is probably a good place to start.

http://www.physics.ohio-state.edu/~wilkins/computing/benchmark/stepanov.html
http://www.research.att.com/~orost/bench_plus_plus.html

-benjamin