testcases [was: Warnings about rcs_id strings: let's settle this]

[Joseph S. Myers]

On Sat, 3 May 2003, Michael Matz wrote:

> >    If the test framework permits, you should automate these tests and
> >    add
> 
> "you _should_".  Not "you _have to_".  Can't we be a bit more ...
> pragmatic to get 3.3 settled?

And above contribute.html says:

   Every patch must have several pieces of information before we can
   properly evaluate it:

Again, a proposed behaviour change can be evaluated better on the patches
to the documentation and the new testcases, with no code, than on code
without testcases and documentation.

This isn't a corner case of doubtful testability or value of testing; it's
a straightforward case where an undesired change to the compiler's
behaviour happened and there should be a test - covering the various cases
fixed by the patch - to stop that change happening again.  "Should" is
only intended to understand that there may be cases where the reason for
the lack of a testcase is obvious or implied, or a test could be automated
but cannot be included for legal reasons, not to suggest that testcases
aren't required where possible.  Again, codingconventions.html describes
what are meant to be invariants of the source tree and says at the top:

      Some existing code may not follow
   these conventions, but they must be used for new code.

(note "must").

(The procedural documentation - contribute.html, codingconventions.html,
etc. - isn't intended for word-by-word dissection of "should"/"must"
anyway - it's intended that it simply be followed as obviously a good
idea, or, if you think it is unclear or there is something wrong with it,
that you propose a patch.)

-- 
Joseph S. Myers
jsm28@cam.ac.uk