Re: Two questions of gcc testing

[Tim Prince <tprince at computer dot org>]

Paul Li wrote:
> I've got two questions of gcc testing.
> 1. gcc has different configurations. how do the test cases address the
> configurations? Is there any coverage criteria for the configurations?
Some of the cases have platform-specific options or exceptions written
into them.  They may be marked as expected pass or expected fail for
certain targets.  Go look.  Secondary targets (e.g. cygwin) aren't
always permitted to use these options, even where other targets use the
same options.  This inflates the reported failures.
> 2. how do the test cases get the oracles (i.e. how to decide the
> testing output is Pass, Fail, ...)?
Generally, the case is set up to return success or failure to the
caller, usually by conditionally executing abort().  Go look.  In
addition, failure to reach the end of the test function (time outs,
crashes, ....) should be caught.