This is the mail archive of the
mailing list for the GCC project.
Re: comparing parallel test runs
- From: Richard Biener <richard dot guenther at gmail dot com>
- To: Aldy Hernandez <aldyh at redhat dot com>
- Cc: GCC Mailing List <gcc at gcc dot gnu dot org>
- Date: Wed, 17 May 2017 13:46:59 +0200
- Subject: Re: comparing parallel test runs
- Authentication-results: sourceware.org; auth=none
- References: <firstname.lastname@example.org>
On Wed, May 17, 2017 at 12:23 PM, Aldy Hernandez <email@example.com> wrote:
> Hi folks.
> I've been having troubles comparing the results of different test runs for
> quite some time, and have finally decided to whine about it. Perhaps someone
> can point out to whatever I may be doing wrong.
> I generally do "make check -k -j60" on two different trees and compare the
> results with:
> cd /dir/current-build-tree
> /gcc-sources/contrib/compare_tests /dir/previous-build-tree .
> However, this gives false positives virtually every time:
> New tests that FAIL:
> g++.dg/tree-prof/pr79259.C execution, -g
> gcc.dg/tree-prof/ic-misattribution-1.c execution, -g
> gcc.dg/tree-prof/merge_block.c execution, -g
> gcc.dg/tree-prof/section-attr-1.c execution, -g
> gcc.dg/tree-prof/section-attr-3.c execution, -g
> New tests that PASS:
> g++.dg/tree-prof/partition1.C compilation, -g
> g++.dg/tree-prof/partition2.C compilation, -g
> g++.dg/tree-prof/partition3.C compilation, -g
> g++.dg/tree-prof/pr79259.C compilation, -g
> Old tests that passed, that have disappeared: (Eeek!)
> g++.dg/tree-prof/indir-call-prof-2.C compilation, -g
> g++.dg/tree-prof/morefunc.C compilation, -g
> g++.dg/tree-prof/pr63581.C compilation, -g
> gcc.dg/tree-prof/20050826-2.c compilation, -g
> gcc.dg/tree-prof/bb-reorg.c compilation, -g
> More often than not the tests that get mentioned are in the gcc.dg/tree-prof
> directory, but I've seen failures in gcc.dg/tree-ssa, gcc.dg/vect and
> When I run these tests manually (make check-gcc
> RUNTESTFLAGS=tree-ssa.exp=whatever.c), they succeed.
> Running with smaller -jNN usually yields less false positives, but rarely do
> I get a clean run.
> This has been happening for as long as I can remember (which may just be 12
> months ;-)).
> Has anyone seen this behavior? Is it my test box? Are there known problems
> with parallel checks?
> FWIW, I'm running on an x86-64 Fedora 25 running a 4.9.6-200.fc25.x86_64
> kernel. The machine has 56 cores, and has dejagnu 1.6 installed.
Likely auto-fdo, see Andis recent patch to imrpove the siutation a
bit. It helped
for me (but I'm only using -j6 on the only machine I see the issue).
> p.s. Attached is the full output from contrib/compare_tests.