Re: proposal for improved management bugzilla priorities/release criteria

[Richard Guenther <richard dot guenther at gmail dot com>]

On Fri, Feb 6, 2009 at 5:30 PM, Paolo Bonzini <bonzini@gnu.org> wrote:
> The current system for managing bugzilla priorities has a major problem,
> in that it does not identify bugs that reasonably cannot be fixed before
> the release.
>
> The current set of priorities is in practice like this:
>
> - P1: most wrong code bugs, and other absolutely blocking problems
> - P2: problems worth a look on important platforms
> - P3: uncategorized
> - P4: problems worth a look on less important platforms
> - P5: other
>
> The problem with this set is that while P1 bugs will absolutely be fixed
> before the release (and backported usually), P2 bugs are a one-catch-all
> group for everything else that's worth looking at.  It is impossible to
> distinguish stuff that will probably be fixed before the release (and
> presumably backported to all branches), and what instead requires new
> stage1/stage2 material.
>
> As a result, the release criteria we have are not really a measure of
> the quality of the release, and especially are not really a measure of
> the work being done towards a release.

I also experience it as limiting that for "important" regressions we only
have P1 and P2 (P3 counts as un-prioritized, so isn't available).

> I propose two solutions to this problem.
>
> - The more conservative one is to use more aggressively the release
> milestone field.  Hard-to-fix bugs would be left as P2, but bumped to
> the next major release at the beginning of stage 3.
>
> Advantages: no need for churn in the bug database---very easy to implement
> Disadvantages: the milestone field is not visible in search lists (maybe
> this can be changed)?

I think using the milestone will get us more confused only.  We already
have the issue that what we make a blocker (P1) for 4.4 is not a blocker
for, say, 4.3.4.  Unless we want to start duplicating bugs for each open
branch I'd rather not touch our target milestone policy.

> - Alternatively, we could add a new priority "P--" for uncategorized
> bugs, and split P2/P3 like this: P2 bugs will be fixed in stage 3/4, P3
> bugs will most likely be postponed to stage 1/2.
>
> Advantages: quicker impression from the bug searches, especially during
> bug triage
> Disadvantages: need to rethink bugzilla queries

We can simply rename the existing P3 to -- (I like no leading P) and add
a new priority P3.  -- would be the new default then.

P3 would then be used for important regressions that do not qualify
for fixing during "stage4" (which includes release branches).  The
advantage over downgrading them from P2 to P4 at the start of
stage4 would be that the distinction is visible earlier and so people
who care can pick ones they find important and make sure they get
fixed before stage4.

> I think any of these two approaches would provide a serious added value
> to judging a release quality.  Meeting the release criteria ("no more
> than 50 P2 regressions") in the past included the release managers
> downgrading bugs from P2 to P4, which is in my opinion cheating.  In the
> proposed scheme, this would be less necessary, because the release
> criteria could take into account a broader view, such as respectively
> for the two approaches:
>
> - At most 60 P2 regressions, of which at most 15 should have release
> milestone 4.4.0.
>
> - No more than 15 P2 regressions and 45 P3 regressions.

I think the only reasonable release criteria is zero P1 regressions over
some period.  50 P2 regressions doesn't make a release blocker, neither
is 49 P2 regressions a clear sign for a non-blocked release.

Richard.

> Any opinions?
>
> Paolo
>
>