testing consistency

Joe Buck jbuck@synopsys.com
Thu Sep 4 18:48:00 GMT 1997


I think that I'd like to see a bit more discipline in how we do testing.
Many of us are doing various tests now, but I do want to make sure that
we cover the cases that users are likely to use.

This means that we should make sure to test the compilers that are built
by following the (yet to be written) instructions exactly, that is,
bootstrap the compiler and build with "-O2 -g", then doing tests with
"clean" environments (nothing else installed: e.g. no libg++ if it isn't
part of egcs).  Additional tests with higher optimization levels and
extra headers are welcome but should be considered "extra".

Ideally, it should be extremely simple to run a "standard test" and
do a full report of results, and mail it back to a standard address.
e.g.

./configure
make
make test_and_report

The idea is that this runs all the test suites, collects all of the
relevant data, and sends it in a mail message.  It would be nice if
cruft is deleted so the mail message isn't full of garbage.

Relevant data that we wouldn't want to miss includes:
	configure options (including platform)
	libc version, if relevant
	assembler and linker versions, if relevant
	gcc options used to compile the final gcc (e.g. -O2 -g)

It may be hard to collect that automatically -- maybe we can just have
a standard "test report template".  It could even be a WWW form.

This process shouldn't be too bureaucratic, but I'd like to be able to
say with confidence that "the following list of platforms passes all
tests" (OK, maybe there is a small set of known failures).





More information about the Gcc mailing list