Re: Power of the G++ test suite

[Joe Buck <jbuck at synopsys dot com>]

> Maybe a bug(-report) tracking system would help, I see lots of people
> who report bugs and get disappointed when they never hear anything
> again...

I agree that this is frustrating.

> A webpage that allows them to find back their report and
> its status might motivate them to contribute more in the future, while
> it will make coordination of pending bugs and fixes easier.

This will only be workable if we have volunteers to go through the reports
and filter them, rejecting the 80% or so that are not real bugs so that
the maintainers don't have to waste their time.

But if it is done:

> 1) Received (automated)
> 2) Format approved
>    or
>    Send back to sender with request to conform to <url>
>    (ie minimize the code; make it standalone/reproducable etc).

This one is really "is this a complete bug report"?

> 3) Confirmed to fail
>    or
>    Doesn't fail with version/environment ... (mail back to sender)

There's another possibility: the report is on platform foo, but there is
no failure on any platform the volunteers have.

> 4) Confirmed to be a real, new bug
>    or
>    Closed: not a bug because <reference to standard>

In practice a lot of bugs will go straight from 2 to 4.

> 5) Confirmed not to overlap with existing tests
>    or
>    Closed: Identical with ...

In practice, it won't be possible to confirm that a bug is not a
duplicate.  It will sometimes be possible to determine that it is,
if we're lucky.

> 6) Code minimized

No, we shouldn't ask volunteers to waste their time trying to change
a test case into a smaller one (unless it is easy to do so).

> 7) Being fixed by ...

Sorry, we shouldn't tell the public which person is responsible for their
bug report unless they are paying $$$ for this service.  (Perhaps such
information might be made available on a private list somewhere, but if
it is public whoever is named will be harrassed: "Why haven't you fixed
my bug yet?").

> 8) Fixed <url to patch>
> 
> Of course it should be garanteed that every bug report is
> handled, and thus a priority system should be set up which
> determines what needs to be done next.

Well, we can make sure that we get to step 1 with every report.

  This priority system
> could depend on date of submission and a 'urgency tag' that
> is assigned during phase 3) for instance.

The above is not unreasonable, but you do need to realize that it's
not going to happen unless someone steps forward and donates the work.
To make this workable, we'll probably need to re-use some existing
piece of code (e.g. PRMS/Gnats or some other system).