GCC testing questions

[Gary Funck]

On Apr 24,  5:57pm, Mike Stump wrote:
> Subject: Re: GCC testing questions
> > From: gary@Intrepid.Com (Gary Funck)
> > Date: Mon, 24 Apr 2000 17:18:38 -0700
> 
> > We're doing some work with an older 2.7.2 based GCC
> 
> Let me urge you to re-port and move up to a more current release.
> 2.95.2 is the current release.
> 
> > The client using the compiler asked how the compiler is tested, and
> > we noted that we had run version 1.44 of the c-torture tests because
> > those were the ones available at the time.
> 
> This is generally my take on testing as well.  However, some would
> also include the notion that you should test with the most current
> testsuite, and go though those results so that you can see all the
> known to current date failures in your compiler, instead of just the
> known some old date failures.
> 
> The questions you ask (they ask) are in general hard to answer, but
> they are good questions.  I don't even have good answers for them.
> I use the metric, <30 getting close, >400 total failure, ~100 still
> needs lots of work.  Personally, if I were release manager for gcc, I
> would rather use the phrase == 0, ok, != 0 needs work, and then just
> pony up the work required to make this happen, but as I know all too
> well, getting the numbers down is a really, really hard job that takes
> years after you decide to try.

Thanks, Mike.  Those are good suggestions.  It won't be
possible moving to the latest GCC because the current
build of the product passes its tests, and going with
new tools would potentially increase risk, and extend
the schedule.  Your answer spoke to what level of passing
might be considered adequate (ie, less than 10 failures),
but not exactly what standard is used by the GCC community
to determine when the base compiler is "done" and is
ready for deployment on the major platforms (what _are_
those major platforms? <g>).