GCC testing questions

[Mike Stump]

> From: gary@Intrepid.Com (Gary Funck)
> Date: Mon, 24 Apr 2000 17:18:38 -0700

> We're doing some work with an older 2.7.2 based GCC

Let me urge you to re-port and move up to a more current release.
2.95.2 is the current release.

> The client using the compiler asked how the compiler is tested, and
> we noted that we had run version 1.44 of the c-torture tests because
> those were the ones available at the time.

This is generally my take on testing as well.  However, some would
also include the notion that you should test with the most current
testsuite, and go though those results so that you can see all the
known to current date failures in your compiler, instead of just the
known some old date failures.

The questions you ask (they ask) are in general hard to answer, but
they are good questions.  I don't even have good answers for them.
I use the metric, <30 getting close, >400 total failure, ~100 still
needs lots of work.  Personally, if I were release manager for gcc, I
would rather use the phrase == 0, ok, != 0 needs work, and then just
pony up the work required to make this happen, but as I know all too
well, getting the numbers down is a really, really hard job that takes
years after you decide to try.

Good luck.