This is the mail archive of the
gcc-patches@gcc.gnu.org
mailing list for the GCC project.
Re: [testsuite] Increase gcc.dg/pr43058.c timeout
- From: Mike Stump <mikestump at comcast dot net>
- To: Rainer Orth <ro at CeBiTec dot Uni-Bielefeld dot DE>
- Cc: gcc-patches at gcc dot gnu dot org
- Date: Fri, 23 Jul 2010 10:30:21 -0700
- Subject: Re: [testsuite] Increase gcc.dg/pr43058.c timeout
- References: <yddr5j6ysjb.fsf@manam.CeBiTec.Uni-Bielefeld.DE>
On Jul 14, 2010, at 8:47 AM, Rainer Orth wrote:
> The gcc.dg/pr43058.c test times out on most of my systems:
>
> WARNING: program timed out.
> FAIL: gcc.dg/pr43058.c (test for excess errors)
>
> Even on an idle Sun Fire T5220 (1.2 GHz UltraSPARC-T2), it takes
>
> real 4:56.38
> user 4:54.71
> sys 0.35
>
> or on an Sun Fire X4450 (2.93 GHz Xeon X7350)
>
> real 1:18.01
> user 1:17.20
> sys 0.26
>
> As soon as the machine is loaded (e.g. make -j<2 * ncpu> check), the
> test is practically guaranteed not to complete within the regular 5
> minute (300 s) timeout. I'd therefore like to increase the timeout by a
> factor of 4.
>
> Ok for mainline and the 4.5 branch?
No. I think the patch is wrong. I think the patch is wrong because the testcase is wrong. The testcase is wrong, as it intentionally consumes tons of resource to try and blow the machine out of the water.
I think 43058 should be re-opened.
Philosophy, any testcase that takes more than 30 seconds on a slow machine, is in danger of being a bad testcase. Really, they should be trimmed, reduced or split up, if possible. The style of this testcase sucks. At least the limit should be lowered to 10 seconds and the testcase reduced to take around 5 seconds on a slow machine (1 GHz say), if people were happy with how slow the compiler is.
A better fix would be to design an extension to the testing infrastructure, say:
PERF: INT test_case_name
and then we could put things like memory usage of the compiler as:
PERF: 121883 RAM gcc.dg/pr43058.c
and compile time performance as:
PERF: 312334 comptime gcc.dg/pre43058.c
where the number is, say, the number of cycles the compiler spent compiling the testcase. We can then add a tool to compare two runs (a la contrib/compare_tests) and report regressions in any performance numbers. Works for RAM usage, paging, compile time, test case run time, number of cache misses, number of spills...
If people like the slow compiler, then the size of the testcase should just be reduced (and the limit dropped), if they don't want to add the infrastructure to do better.