Re: GCC2 merging (was "native language support now available")

[joel at OARcorp dot com]

On Tue, 13 Oct 1998, Richard Kenner wrote:

> You mean overall testing.  People in the software engineering community often
> say that such testing isn't very valuable.  The kind of testing that's the
> most important in producing high reliability software is *unit testing*,
> which means testing each function individually to make sure it meets its
> specifications under all conditions.  As far as I know, this is not done for
> either gcc2 or egcs.

I agree with this.  But from a practical viewpoint, frequently running 
regression tests on a variety of host/targets is important even if you
have unit tests.

It is also difficult to build up unit tests on a large body of software
after the fact.   I think it would be nice if such tests existed and could
be run automatically but unfortunately they do not exist today.

> An interesting exercise would be to look at the ISO 9000 standard and see if
> either project meets it and to try to figure out what SEI level each project
> has.

I think it would be interesting to see how these standards could be
applied to such a distributed, volunteer based development effort.  There
are certainly benefits to ISO 9000 and SEI levels.  Both require some
administrative infrastructure and controls which would be challenging to
have in a distributed environment.  Interesting.

> It is possible to derive *some* information from bugs, but what you have to
> look at is not the number of bugs but the nature of each in detail.  I'm not
> that familiar with the work that's been done in that area.

Yes.  If you get 12 reports of the same bug, only one good test case
should enter the regression tests.

On a related note, bugs should be tracked by subsystem or module so a
piece of the implementation with high defect rates can be redesigned.
I have seen a handful of articles which claim this type analysis and
replacement of problem code can be highly beneficial.

--joel