This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: A testcase library
- From: Jason R Thorpe <thorpej at wasabisystems dot com>
- To: Volker Reichelt <reichelt at igpm dot rwth-aachen dot de>
- Cc: jakub at redhat dot com, gcc at gcc dot gnu dot org, bangerth at ticam dot utexas dot edu,janis187 at us dot ibm dot com, ehrhardt at mathematik dot uni-ulm dot de
- Date: Fri, 20 Dec 2002 09:32:08 -0800
- Subject: Re: A testcase library
- Organization: Wasabi Systems, Inc.
- References: <20021220125202.B1310@sunsite.ms.mff.cuni.cz> <200212201313.OAA72149@numa6.igpm.rwth-aachen.de>
On Fri, Dec 20, 2002 at 02:13:23PM +0100, Volker Reichelt wrote:
> =================================================================================
> On Fri, Nov 29, 2002 at 09:47:24AM +0000, Nathan Sidwell wrote:
> > Craig Rodrigues wrote:
> > > A few questions:
> > > (1) Can I commit my testcase to the testsuite, since it is definitely
> > > a regression from gcc 3.1 and gcc 3.2?
> > No, we only commit test cases when the bug's fixed. The rationale being you
> > don't want to add a new failing test case -- that'd confuse people. Marking
> > it XFAIL isn't really the right solution, as then when you fix the bug, you
> > don't know whether fixing the XFAIL was accidental, and end up adding a new
> > test case anyway. Also, in fixing the bug, you (well I at least), can
> > construct a better test case having undersytood the failure mode.
> =================================================================================
I have to say that I disagree with this policy, esp. if the testcase shows
a regression from a previous GCC release. Gaps in the testsuite mean it
will take longer for bugs to be fixed, and give people a false sense of
confidence in the quality of the compiler when they see results that show
few, if any, failures.
One example is a PR I filed recently, with test case that causes the compiler
to loop forever. The rationale I got for not adding the testcase was "it
would make everyone's bootstrap/test cycle 5 minutes longer". My initial
thought was "...and that will motivate someone with the knowledge to do so
to fix it." (The test case in question shows a regression from a previous
release of GCC.)
...the principle is not too different than a suggestion someone made a
while ago about making GCC maintainers do bootstraps on 100MHz Pentiums
(maybe GCC wouldn't slow down so much over time if they did :-)
--
-- Jason R. Thorpe <thorpej@wasabisystems.com>