Re: Why not gnat Ada in gcc?

[Mike Stump <mrs at windriver dot com>]

> To: dewar@gnat.com, law@redhat.com, mrs@windriver.com
> Cc: gcc@gcc.gnu.org, kenner@vlsi1.ultra.nyu.edu, rms@gnu.org, rth@cygnus.com
> Date: Wed,  1 Nov 2000 23:39:41 -0500 (EST)
> From: dewar@gnat.com (Robert Dewar)

> Presumably only Cygnus/Redhat can measure this, since only they have
> the internal test suite that would give the data. 

:-) Nope, on a couple of counts.  First, my SOP was to take the time
to sanitize the testcase, and fold it into the testsuite.  Nearly all
of my testcases made the transition from company confidential to
public testcase.  I don't know exactly how many were lost in the
transition to publicness, but my guess is not that many for the g++
testsuite.  (<3%?)  For the gcc testsuite, quite a bit of the content
originated as a public testsuite (c-torture).

Second, it can still be measured against the public testsuite, even
though some internal testsuites might give slightly better results.

Third, the there is a testresults web page that has the last
information on it (http://gcc.gnu.org/testresults/), you merely have
to mine it out.

And last, _you_ have a testing framework that can tell you about the
quality of gcc.  You merely have to ask it.  (Yes, I know how hard
this is.  I have my own framework as well, complete with ~88 boards),
and I don't yet ask it about public sources.)

> Well I think in the case of Ada it will be a huge help to make the
> official validation (ACATS) tests available and runnable, and that
> is one of our projects (these are openly available tests, but
> setting them up to run is nowhere near trivial).

Should be.  For gcc/g++/g77/objc/libstdc++/libio, the goal is make
check.  That mostly works.  It is mostly easy to run.  If you engineer
them to work from make check, I think they can be made to be trivial.
For example, the Fortran testsuite is so trivial to run, that almost
no one who runs tests doesn't also run it, even though they aren't
typically interested in Fortran.

> Unfortunately the more significant test suites (the ACT internal
> test suite, and the Digital test suite) are full of highly protected
> proprietary code, and cannot be made available for external use.

Can you report the results publicly?  If so, maybe you can give warm
fuzzies, without saying exactly what went wrong.

> It will be interesting to see how stable things stay ...

It'll be 6 months to condition people to understand the results you
feed them, and to condition them to stop breaking it.  After the ramp
up, people should mostly keep it to the quality you expect.  If they
don't, let's talk about it again, then.