This is the mail archive of the
mailing list for the GCC project.
Re: What is a regression?
- From: Mark Mitchell <mark at codesourcery dot com>
- To: Ian Lance Taylor <iant at google dot com>
- Cc: Jason Merrill <jason at redhat dot com>, GCC <gcc at gcc dot gnu dot org>
- Date: Tue, 23 Oct 2007 11:53:13 -0500
- Subject: Re: What is a regression?
- References: <471CFD3A.email@example.com> <firstname.lastname@example.org>
Ian Lance Taylor wrote:
> Jason Merrill <email@example.com> writes:
>> I think that the release process for recent releases has given undue
>> priority to bugs marked as regressions. I agree that it's important
>> for things that worked in the previous release to keep working in the
>> new release. But the regression tag is used for much more trivial
> We had a discussion of these sorts of issues at the GCC mini-summit
> back in April. We didn't come to any conclusions.
There's a misconception in this thread that I need to address as the RM:
the notion of "release-blocker". We don't really have release blockers,
because we don't have a way of making sure things get fixed. If I
declare a bug a release blocker, we may just never have another release.
Companies can declare something a release-blocker because they can then
direct resources to go work on the issue. We don't have that luxury.
When I look at the bug list immediately before making a release, I scan
the P1s and decide if I can live with myself after I push the button.
When I mark a PR as "P1", that means "This is a regression, and I think
it's embarrassing for us, as a community, to have this bug in a
release." Unfortunately, every release goes out with P1 bugs open, so
we can't really call them "release blockers". My judgment isn't always
great, and it's certainly not final: I'm willing for people to suggest
that P1s be downgraded. I've suggested that people do that by putting a
note in the PR, and CC'ing me.
I agree that PR32252 (the example given by Jason) should not be P1.
I've downgraded it to P2. (P2 means "regression users will notice on a
major platform, but not P1". P3 means "not yet looked at". Things like
ICE-after-valid-error are marked P4. Things utterly outside the release
criteria are P5.)
One of the consistent problems with GCC has been that we disregard the
experience of users that aren't power users. However, I disagree that
these things should be extremely low priority. Compiler ICEs send
message that what we're producing is not of high quality so I think we
should take them seriously.
I do also think we should start weeding out some of the P2s that have
been around a long time. If GCC 2.95 didn't ICE, but GCC 3.0+ have all
ICE'd, then, at this point, it's not much of a regression any more; it's
just an ICE that's "always" been there.
I do think that we should take regressions, in general, very seriously
-- especially cases where we generate wrong code. One of the consistent
complaints I have heard is that people fear GCC upgrades and the
perception is that this is worse than with other compilers. Certainly,
I've talked to other compiler vendors who claim that they consider any
wrong-code generation bug a release blocker.
Now, all that said, of course I think that other bugs are important too,
and I'm all for fixing them. But, in terms of looking at a release and
deciding how ready-to-go it is, I think regressions are as reasonable a
measure as any.
(650) 331-3385 x713