[testsuite] Run guality tests on Solaris

[Rainer Orth]

Jakub Jelinek <jakub@redhat.com> writes:

> On Thu, Jan 29, 2015 at 11:11:12PM -0700, Jeff Law wrote:
>> >                                 Linux/x86_64 Solaris 11/x86 Solaris
>> > 11/SPARC
>> >                                 (Fedora 20)
>> >
>> >gcc.dg/guality:
>> >
>> ># of expected passes            6490		6500		5489
>> ># of unexpected failures        191             171             802
>> ># of unexpected successes       61              66              73
>> ># of expected failures          35              30              23
>> ># of unsupported tests          257             267             383
>> >
>> >g++.dg/guality:
>> >
>> ># of expected passes            128             128		118
>> ># of unexpected failures        6               10              10
>> ># of unsupported tests          34              30              40
>> Could we select a version of gdb as a reference version, then xfail those
>> tests which don't work with that reference version on each platform to cut
>> down the noise?  Obviously we bump the reference version, probably as we
>> close stage1 or stage3 development?
>
> The biggest problem is that what fails and what does not varries between
> targets and between optimization levels.  Right now we have no way to xfail
> test XYZ for -Os on x86_64-linux and for -O2 and -O3 on i686-linux ia32, and
> the lists would become very large.  Some tests in guality are xfaileded just
> in case, even when they actually XPASS on many targets.

Right: while we could add such a facility (the current xfail selector
support is already beyond what vanilla DejaGnu provides), it would
quickly become very unwieldly.  The best we can do right now is to use
dg-xfail-run-if more precisely for tests that fail to execute.  For
gdb-test failures, the only option would be to dg-skip-if the affected
tests per target and optimization level; quite large a hammer in some
cases.  I've started looking at this, but it's incredibly tedious.

> The way to look for regressions in the guality area, at least as I do it
> regularly, is just compare test_summary results.
> If we'd disable this by default, I'm sure our debug quality would sink very
> quickly.

I'm not sure: my impression is that nobody really cared about the
guality tests in quite some time.  Comparing test results might work
resonably, looking over the output if the vast majority of it is just
guality failures becomes infeasible.  I may end up running with
GUALITY_GDB_NAME=/bin/true ;-(

	Rainer

-- 
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University