Re: [testsuite] Run guality tests on Solaris

[Rainer Orth <ro at CeBiTec dot Uni-Bielefeld dot DE>]

Jakub Jelinek <jakub@redhat.com> writes:

> On Wed, Jan 28, 2015 at 01:42:47PM -0700, Jeff Law wrote:
>> >2015-01-28  Rainer Orth  <ro@CeBiTec.Uni-Bielefeld.DE>
>> >
>> >	gcc/testsuite:
>> >	* gcc.dg/guality/guality.h (main): Add argv[0] to
>> >	guality_gdb_command.
>> OK.
>> 
>> As for what to do with guality, I haven't a clue.  They're dependent on the
>> debugger version and perhaps other stuff that I don't recall.
>> 
>> Perhaps skip them if we find gdb and determine it is "too old"?
>
> We already do that.  I bet the Solaris case is more about the lack
> of support to find an executable from its pid (/proc/<pid>/exe
> in Linux).
> Guess one can easily try it, run
> gdb
> (without arguments, or those -nx -nw that guality uses) and type
> attach 19355 # pick pid of some process you can debug
> in Linux gdb will find the binary etc.

That issue is easily solved by passing the executable name to gdb; this
is guaranteed to work everywhere.  On Solaris (at least from Solaris 10
onwards, haven't checked earlier version), gdb could use
/proc/<pid>/path/a.out to get at the executable, but that won't help for
released versions of gdb (and eventually other platforms which provide
no such facility).

But this issue is minor and easily avoided.  The major problem is that
on both Solaris and Linux, many of the guality tests FAIL (or XPASS,
equally adding noise to mail-reports.log) even with a current version of
gdb (7.8 in my case):


                                Linux/x86_64	Solaris 11/x86	Solaris 11/SPARC
                                (Fedora 20)

gcc.dg/guality:

# of expected passes            6490		6500		5489
# of unexpected failures        191             171             802
# of unexpected successes       61              66              73
# of expected failures          35              30              23
# of unsupported tests          257             267             383

g++.dg/guality:

# of expected passes            128             128		118
# of unexpected failures        6               10              10
# of unsupported tests          34              30              40

It also seems (haven't checked yet in detail) that the results also
depend on whether they are created as part of a regular make check at
the toplevel, compared to runtest --tool <tool> guality.exp.

Judging from posted testresults, I'm no the only one seeing this, and
the guality tests have way more FAILs than all other tests combined:
with those amounts of noise, it's almost impossile to see other errors,
and nobody seems to work on fixing those.

Thus my suggestion not run them by default until someone steps forward
to take care of all those issues.

	Rainer

-- 
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University