Re: Bug tracking / Release Quality Assurance

[Roger Sayle <roger at eyesopen dot com>]

Gabriel Dos Reis writes:
> Gerald Pfeifer <pfeifer@dbai.tuwien.ac.at> writes:
> | Checking the GNATS database which currently shows 62 high priority
> | problems, there are some points I'd like to raise:
> |
> | 1. I believe we should not ship GCC 3.2.1 (or branch 3.3.) until this
> |    bug count has been significantly reduce.
>
> Agreed.
>
> Seems like, somewhere at some point, our development model failed to
> cope with only-bug-or-regression fixes and improving (in the broad
> sense) the development branch: Balance interests in fixing bugs and
> adding new functionality.  Clearly closing all branches won't
> automatically make people fixing bugs, but somehow we need to improve
> on the current situation.

[Lots of my very humble opinions...]

I'd like to offer an alternative perspective.  I think 62 high priority
PRs in GNATS is very impressive given that there were 75 just over a week
ago.  At this rate there'll be no outstanding high priority PRs by the
3.3 release! :>

The "obvious" source of the problem is that the rate at which bugs are
submitted to GNATS is significantly higher than the rate at which they're
getting fixed.  I've been following the graphs since April...

Table of non-closed PRs in GNATS by date (row) and category (column)...

date     all  boot c&++ c++  dbug g77  java lgcj mid  objc opt  othr stdc targ
20020430 1256  126  591  471   12   10   77   46   16   15   85   86   45  102
20020502 1262  127  596  478   11   11   78   44   16   15   86   86   42  102
20020508 1288  129  605  482   12   11   75   45   18   15   87   87   45  102
20020514 1301  130  606  483   12    9   75   46   19   15   94   91   49  104
20020519 1320  129  620  496   12    9   76   46   19   13   97   89   48  107
20020523 1355  134  623  499   12   10   77   48   19   13  102   93   58  109
20020527 1383  139  638  505   12   10   81   49   18   13  107   95   55  108
20020601 1418  143  656  512   12   10   80   49   18   13  114   98   50  113
20020607 1451  148  658  510   12   10   79   49   20   18  118  101   48  120
20020612 1475  148  668  518   12   10   77   49   21   18  128  102   50  121
20020616 1502  157  677  520   12   10   77   49   23   17  132   88   52  133
20020626 1504  157  694  536   14   11   81   51   22   17  133   91   56  103
20020702 1519  164  696  537   14   11   80   49   22   16  140   91   60  104
20020721 1591  169  720  538   18   12   84   49   29   11  150   98   61  113
20020802 1640  173  741  553   18   13   84   50   29   11  156  100   58  122
20020826 1787  189  783  610   24   15   84   56   47   11  176  110   60  154
20020908 1885  198  838  652   24   15   90   60   48    9  178  116   67  160
20020928 1940  214  829  644   26   15   91   58   55    5  186  122   79  170
20021005 1908  217  828  644   29   14   91   60   53    5  175  124   77  156

As you can see with the exception of the hard work of Nicola Pero (and
Stan Shebs?) to bring down the objc numbers most other columns are out
of control.  But notice also the recent effects of Mark Mitchell's
requests to fix bugs (or threats to close branches) has resulted in
the first significant drop in the total number of PRs in 6 months.

As a contributor, virtually all high priority optimization PRs have
now been fixed or are assigned to someone (7124 == 7426, 7034 == 6845).
But by far the largest category of high priority PRs is C++.  My numbers
from the weekend before last are 28 C++, vs. 8 optimization, 8 target,
7 C with all other categories 4 or less.

By their very nature C++ bugs are difficult to fix.  Hence GCC forked
a C++ parser branch to tackle many of these problems in the medium to
longer term.  I believe the current problems are a side-effect of the
decision between short-term fixes in the current C++ infrastructure
vs. longer-term fixes in the new parser.

[I could go into a longer analysis of why the current situation is
as it is, but suffice to say the problems couldn't be avoided and
everyone involved is doing the best they can].

One possible improvement in the current situation would be for the C++
maintainers to review patches as quickly as possible.  Patches to most
other parts of the compiler are reveiwed by RTH at a phenomenal rate,
but my impression is that C++ changes/fixes are typically approved by
someone else and therefore take slightly longer. [I've absolutely no
data to support this claim].  However reducing this negligible barrier
to C++ patches may improve the overall dynamic.

And I really hate to say it, but a longer term solution might be better
C++ internals documentation.  This would reduce the effort required to
become a g++ front-end wizard.  Ultimately, g++'s problems can't be
fixed by making the maintainers work harder.  What's needed are ways
to get more volunteers to offer solutions.

Roger
--
Roger Sayle,                         E-mail: roger@eyesopen.com
OpenEye Scientific Software,         WWW: http://www.eyesopen.com/
Suite 1107, 3600 Cerrillos Road,     Tel: (+1) 505-473-7385
Santa Fe, New Mexico, 87507.         Fax: (+1) 505-473-0833