Re: [testsuite] C++ i386 ICE

[Jeffrey A Law <law at cygnus dot com>]

  In message <20000614082211U.mitchell@codesourcery.com>you write:
  > >>>>> "Jeffrey" == Jeffrey A Law <law@cygnus.com> writes:
  > 
  >     Jeffrey> That's fine too, but I really like the capability of
  >     Jeffrey> being able to run the testsuite and get a list of
  >     Jeffrey> failures that we would consider regressions for a
  >     Jeffrey> release.
  > 
  > That's a good idea -- but not in general possible.  You have to look
  > in GNATS now, because some bugs are in GNATS without corresponding
  > test cases in the testsuite tree.  (That's actually the "normal"
  > situation in most organization, AFAIK.  QA and customer support add
  > things to the bug database; engineers make regression tests when they
  > act on bug reports from the bug database.)
I disagree with recording this stuff strictly in GNATS, but it's your
decision to make.

One of the best benefits of having this stuff in the testsuite showing up
as a FAIL is that people fix them in a reasonable timeframe.  If they're
sitting in a bugs database or are XFAIL, then they tend to be ignored.
That's based on years of experience with this kind of stuff.

I'm not knocking traditional organizations, but in my experience the quicker
the test goes into the testsuite and gets widespread exposure as a bug, the
quicker it gets fixed.


jeff