This is the mail archive of the
gcc-patches@gcc.gnu.org
mailing list for the GCC project.
Re: parallel check output changes?
- From: Segher Boessenkool <segher at kernel dot crashing dot org>
- To: Andrew MacLeod <amacleod at redhat dot com>
- Cc: Jakub Jelinek <jakub at redhat dot com>, Bernd Schmidt <bernds at codesourcery dot com>, gcc-patches <gcc-patches at gcc dot gnu dot org>, richard dot sandiford at arm dot com
- Date: Thu, 25 Sep 2014 12:02:32 -0500
- Subject: Re: parallel check output changes?
- Authentication-results: sourceware.org; auth=none
- References: <541B1710 dot 8060809 at codesourcery dot com> <20140918173609 dot GM17454 at tucnak dot redhat dot com> <20140918184455 dot GB28595 at gate dot crashing dot org> <20140919093723 dot GA26414 at gate dot crashing dot org> <87iokel5c0 dot fsf at e105548-lin dot cambridge dot arm dot com> <5422DB41 dot 1090800 at redhat dot com> <20140924161039 dot GA27079 at gate dot crashing dot org> <5422F174 dot 2020001 at redhat dot com> <5423065F dot 7020308 at redhat dot com> <54240905 dot 70600 at redhat dot com>
On Thu, Sep 25, 2014 at 08:22:29AM -0400, Andrew MacLeod wrote:
> So to be fair, I could use test_summary, but I think the concern is
> warranted because if this inconsistent ordering can happen to PASS, I
> would expect the same non-deterministic behaviour if those tests happen
> to FAIL. we just have far less FAILS so we aren't seeing it with
> test_summary at the moment...
>
> Aggregating all my .sum files, I see a sampling of about 257,000 PASSs,
> whereas I see a total of 141 FAILs. FAILs only account for < 0.06% of
> the output. ( I'm getting an average of about 510 mis-ordered PASSs, so
> it only affects a small portion of them as well.)
0.24% here (2241 FAILs, 917715 PASSes).
You're seeing about 1 in 500 misordered, so if it was independent (which
of course it is not) I should see it in the FAILs already.
> I would think the output of .sum needs to be consistent from one run to
> the next in order for test_summary to consistently report its results as
> well.
Yes. There also is the problem of the summaries being messed up (which
they were already before the parallelisation changes, but now the result
is much worse).
I'll have another look.
Segher