[v3] Refined test instrument for container tests

Benjamin Kosnik bkoz@redhat.com
Mon Jun 3 03:38:00 GMT 2002


Hmm. I think everybody liked the direction of the original patch, but
that the libstdc++ sources were slushed and major upheavals in the
testing infrastructure made everybody uneasy when day-to-day tracking of
regressions was so important.

I should have said that at the time, I suppose. Sorry, Stephen. Also, I
was looking for a more comprehensive demonstration of using the new test
harness.

By now, it's obvious that there are a couple glaring holes in libstdc++
testing. Some of them include:

- lack of clear distinction between required functionality tests and
so-called extra funtionality tests, such as the named locale tests in
testsuite/22_locale being XFAILED for non-linux targets. I'll need to
write wchar_t io tests in the really near future, which will also have
this issue. Perhaps the way out is illuminated by Ghazi's ctype tests,
and the resulting attempt to split the ctype member functions into
smaller tests, most of which are now enabled. I'm still not clear on how
to name the tests and organize them to prevent confusion and mass
XPASSING/XFAILING: I'd thought about putting all the named locale tests
into the ctype_members_*.cc file, but I dunno. Seems odd.

- lack of integration of interactive io tests

- lack of performance metrics, including time to compile, size of
binaries, speed of execution, memory usage.

- lack of exception and exception-safety testing

- lack of automatic leak testing.

- coverage issues for the STL portions (and seemingly things as simple
as over-riding gobal new.... yikes)

I know there are others. Loren has a separate set of issues, probably.

;)

Granted, this is mostly wishful thinking. It would be fun to experiment
at this point to try and get some solutions in place for some or all of
these points. That's why I liked the intent behind Stephen's patch so
much.

The situation is not completely dire: the libstdc++-v3 tests are still
better and more complete than libstdc++-v2 and improving. Many things
are actually caught by the testsuites.

> It's a good start: I was already working to add the missing (new)
> functionality in the form of more counters so I could submit my
> deque test suite to follow on to your reformatting, but I ran into
> a problem in that I can't build from the trunk because my libc is
> too old, and I can't build a newer libc because there's some kind of
> a bug in the gcc configury such that it's picking up a really old ld,
> and boy am I borrowing against my karma to get things back up and
> running.
> 
> I'll try and get something in the next week or so, since the original
> deque test suite has been kicking around my disk since last December
> and I don't think these things age like wine.

Sounds good.

-benjamin



More information about the Libstdc++ mailing list