[RFC] testsuite/performance

[Benjamin Kosnik <bkoz at redhat dot com>]

Hello people. 

There's long been talk of storing performance-related tests in the
testsuite itself. Here's a first crack at actually doing this. In
preparing this patch, I did a lot of CVS/web/mailing list excavation
(including the finding of Loren's post to switch back to the pool
allocator on 2001-05-03, which it seems is the basis for the subsequent
allocator tests). It was pretty interesting, but perhaps I forgot
somethings. If so, let me know.

I thought it was a good idea to put these sources in the testsuite, even
if they are not run by default, so that the maintainers are not all
carting around little test programs in a separate directory. (As I've
been doing for much too long now.) Thoughts? Suggestions and improvements?

Issues/Notes.

1) Many of these tests are similar, large loops doing one thing.

2) Some of these tests produce large output files (75MB) (which are
removed with make clean, of course.) Some require large input files.
Should they be run by default?

3) There are no hooks to run these. Instead, they are just run and the
results thrown away. It's assumed that people will run these by hand as
part of the release process. (This is a similar situation as the
interactive tests.) Perhaps something like the bench++ timers can be
added later to testsuite_hooks.cc.

4) There is no extension or naming practice designating the performance
tests. Should there be? If so, then they could be mixed into the usual
organization like the _neg tests and the _xin tests, and perhaps ignored
by default. This is probably a better solution than what I've got here.

5) Some of these tests represent problems that have not been fully
addressed. Putting them in the testsuite just means that it's easier for
maintainers to track these issues.

6) Some, like 4150 are not suitable for admission.

I'm in the middle of revamping the testsuite documentation, but thought
I'd post this to see what people have to say.

-benjamin

Attachment: p.20030516-1
Description: Binary data