Request for Steering Committee: official release criteria for 4.x

["Giovanni Bajo" <giovannibajo at libero dot it>]

Hello,

this is a request for the Steering Committee.

I think we need to revive a page holding a list of the official release
criteria for 4.x releases. Right now, it is all pretty arbitrary and heuristic,
with Bugmasters knowing probably more rules than many mainteners (having
deduced those rules, eg., carefully watching the Bugzilla work being done by
RMs, which is something not all maintainers are supposed to do). I would like
such a page to answer [at least] these questions:

- Which are the so-called "primary platforms" for GCC 4.x? [what is,
technically, a primary platform? I would say that a primary platform is
platform whose specific regressions might stop a release. Is this true?]

- Which are the so-called "secondary platforms" for GCC 4.x? [what is,
technically, a secondary platform? I would say that it is a platform for which
broken bootstrap might be a show-stopper for a release. If a bug happens on a
platform which is neither primary nor secondary, should it be targeted at the
next major release automatically or not?]

- Which are the real-world applications that should absolutey be tested as
working? [Kernel, Glibc, POOMA, Boost...]

- Which is the oldest release against which regressions are checked? [2.95
probably]

- Which is the compile-time threshold at -O0 for a bug to be considered a
regression? [used to be 25%]

- Which is the compile-time threshold (if any!) at -O1/-O2/-O3/-Os for a bug to
be considered a regression? [used to be 25%]

- Which is the memory occupation threshold at -O0/-O1/-O2/-O3/-Os for a bug to
be considered a regression?

- Which is the binary size threshold at -Os for a bug to be considered a
regression?

- Which is the binary size threshold (if any!) at -Os for a bug to be
considered a regression?

- Do we want a small set of files/applications to closely monitor code
generation quality and compile time behaviour? The old release criteria
suggested some (LAPACK, Stepanov abstraction test, gzip). We could isolate a
small number of preprocessed (compilable and runnable, where possible) sources
against which run a set of script-based performance tests. For these files,
probably the thresholds for regressions could be smaller (5% or so). I believe
this could be automated in "make check-performance", using another specified
compiler as the baseline (for instance, it could be the bootstrapping compiler
by default).


If these questions are officially answered and/or other criteria officially
specified, I volunteer to do the html legwork to revive the criteria page on
the site.

Thanks
Giovanni Bajo