Re: GCC 4.4.0 Status Report (2008-11-17)

["Steven Bosscher" <stevenb dot gcc at gmail dot com>]

On Mon, Nov 17, 2008 at 11:15 AM, Jakub Jelinek <jakub@redhat.com> wrote:
> Quality Data
> ============
>
> Priority          #     Change from Last Report
> --------        ---     -----------------------
> P1               13     -  4
> P2              114     - 27
> P3                3     +- 0
> --------        ---     -----------------------
> Total           130     - 31
>
> We've made a good progress on fixing P1 and especially P2 bugs, but
> still have many to fix before we reach the criteria for branching
> 4.4.0.  Several of the P2 and P1 bugs have patches posted, but
> not reviewed yet.
>
> All but one P1s are regressions from 4.3 and there are 25 P2 regressions
> from 4.3, especially those should be given attention.

More than half of the regressions have 4.2/4.3/4.4 in the summary
(i.e. regressions from GCC 4.1 or older). It is funny to see
"high-priority" regressions that go unfixed for at least 2 major
releases still counted as measures for the quality of GCC 4.4.

Would it be asked too much if I suggest again that someone should go
through the list of regressions and reduce the priority of some of
them?  I'll list my favorite PRs with a P1, P2 or P3 priority which
should IMHO be dropped to P5 (or P99 ;-):

* PR19079 - quadratic behavior with many sets
Quadratic behavior is never good, but this test case is so unrealistic
that fixing this "problem" doesn't have to be high-priority. This bug
exists since GCC 4.1 but has not been reported by anyone else, that
is, no duplicates for real world code. Fixing this problem would cause
a (probably significant) compile time penalty on real world code.
Might as well close as WONTFIX, but since it's technically a
regression I suppose it has to stay open. But not at P2, if you ask
me.

* PR18219 - bloats code by 31%
Case of a missed optimization for size, fails since GCC 4.0 - but
really, CSiBE scores clearly show that overall, GCC has made huge
leaps forward for code size with GCC 4.0 and later for all targets.
This bugs highlights the problem with using all regressions as a
measure for quality: There is a regression from a "local optimum" and
it is counted as a quality regression, even though the "global
optimum" is approached much more closely with the "regressing"
release. This is probably true for many (most?) of the
"missed-optimization" regressions in the GCC 4.4 regressions list. The
next bug is another example:

* PR28868 - Not eliminating the PHIs which have the same arguments
This is "known to work" in GCC 3.3, but already fails in GCC 3.4.  The
summary is misleading, the PR actually discusses a missed
optimization. Of course, the reason for the missed optimization in GCC
3.4 has nothing to do with PHIs (there were no PHI nodes in GCC 3.4).
Anyway, the reason for this "problem" in GCC 4.x is a design decision
to not replace one SSA_NAME with another if they are equivalent, to
avoid awkward overlaps of SSA_NAME live ranges.
This bug says nothing about the quality of a release of GCC 4.4. It's
just a design choice with a trade-off, where there is a win that is
more important than the negative consequences. This kind of bug should
not be P1/P2/P3, if you ask me.

* PR32306 - DOM jump threading no longer iterates
Another design choice. Iterating DOM was too expensive in terms of
compile time. In Jeff's words from today: "we're on the right side of
the cost/benefit curve"
(http://gcc.gnu.org/ml/gcc/2008-11/msg00224.html). Another deliberate
design choice marked as a regression.


There are more bugs like this (maybe 12, 16 in total or so). I think
that if you keep counting this kind of PRs in the "Quality Data", it
gives a much darker (and, frankly, quite flawed) indication of the
quality of the compiler. In fact, if you judge the state of GCC 4.4
based on the Bugzilla regressions list, it is in surprisingly good
state compared to GCC 4.3 (I don't know how many bugs GCC 4.4 has that
are not regressions, of course).

In business-speak, one would probably say that just the number of
regressions is not a good Key Performance Indicator ;-)

Just my $0.02

Gr.
Steven