[RFC, v2] Test coverage for --param boundary values

[Jeff Law]

On 07/18/2016 01:53 AM, Martin Liška wrote:
> You are right, I was inspired by what we do for GCC plugins in:
> gcc/testsuite/lib/plugin-support.exp
>
> where we have following comment:
>     # Note that the plugin test support currently only works when the GCC
>     # build tree is available. (We make sure that is the case in plugin.exp.)
>     # Once we have figured out how/where to package/install GCC header files
>     # for general plugin support, we should modify the following include paths
>     # accordingly.
>
> Well, I can imaging a guard which will test whether "$objdir/../../params.options" file exits,
> and if so, then the tests are executed? Is it acceptable approach?
Yes.  That's acceptable to me.

>
>> >
>>> >> +set text [read $fd]
>>> >> +close $fd
>>> >> +
>>> >> +# Main loop.
>>> >> +foreach params [split $text "\n"] {
>>> >> +    set parts [split $params "="]
>>> >> +    set name [string trim [lindex $parts 0] '"']
>>> >> +    set values [split [lindex $parts 1] ","]
>>> >> +    if { [llength $values] == 3 } {
>>> >> +	set default [lindex $values 0]
>>> >> +	set min [lindex $values 1]
>>> >> +	set max [lindex $values 2]
>>> >> +	set int_max "INT_MAX"
>>> >> +
>>> >> +	if { $min != -1 } {
>>> >> +	    param_run_test $name $min
>>> >> +	}
>>> >> +	if { $max != $min && $max > 0 && $max != $int_max } {
>>> >> +	    param_run_test $name $max
>>> >> +	}
>>> >> +    }
>>> >> +    if { [llength $values] == 5 } {
>>> >> +	foreach v $values {
>>> >> +	    param_run_test $name $v
>>> >> +	}
>>> >> +    }
>>> >> +}
>> >
>> > This produces ambiguous test result lines:
>> >
>> >     Running /scratch/tschwing/nvidiak20-2/gcc/trunk-light/source-gcc/gcc/testsuite/gcc.dg/params/params.exp ...
>> >     PASS: gcc.dg/params/blocksort-part.c (test for excess errors)
>> >     PASS: gcc.dg/params/blocksort-part.c (test for excess errors)
>> >     PASS: gcc.dg/params/blocksort-part.c (test for excess errors)
>> >     PASS: gcc.dg/params/blocksort-part.c (test for excess errors)
>> >     [many more]
>> >
>> > OK to fix this as follows:
>> >
>> >     PASS: gcc.dg/params/blocksort-part.c -O3 --param predictable-branch-outcome=0 (test for excess errors)
>> >     PASS: gcc.dg/params/blocksort-part.c -O3 --param predictable-branch-outcome=50 (test for excess errors)
>> >     PASS: gcc.dg/params/blocksort-part.c -O3 --param inline-min-speedup=0 (test for excess errors)
>> >     PASS: gcc.dg/params/blocksort-part.c -O3 --param max-inline-insns-single=0 (test for excess errors)
>> >     [...]
>> >
>> > commit ae350ce9af062d7d2c05225280596b1bc982728b
>> > Author: Thomas Schwinge <thomas@codesourcery.com>
>> > Date:   Fri Jul 15 09:13:19 2016 +0200
>> >
>> >     Disambiguate gcc.dg/params/params.exp test results
>> >
>> >     	gcc/testsuite/
>> >     	* gcc.dg/params/params.exp (param_run_test): Swap flags and
>> >     	default-extra-flags arguments when calling dg-runtest.
>> > ---
>> >  gcc/testsuite/gcc.dg/params/params.exp |    2 +-
>> >  1 file changed, 1 insertion(+), 1 deletion(-)
>> >
>> > diff --git gcc/testsuite/gcc.dg/params/params.exp gcc/testsuite/gcc.dg/params/params.exp
>> > index f2f397e..dd82a5d 100644
>> > --- gcc/testsuite/gcc.dg/params/params.exp
>> > +++ gcc/testsuite/gcc.dg/params/params.exp
>> > @@ -28,7 +28,7 @@ proc param_run_test { param_name param_value } {
>> >      global srcdir
>> >      global subdir
>> >
>> > -    dg-runtest $srcdir/$subdir/blocksort-part.c "" "-O3 --param $param_name=$param_value"
>> > +    dg-runtest $srcdir/$subdir/blocksort-part.c "-O3 --param $param_name=$param_value" ""
>> >  }
>> >
>> >  set fd [open "$objdir/../../params.options" r]
>> >
>> >
>> > Grüße
>> >  Thomas
>> >
> Thanks for the patch, that's probably the same problem that Andreas spotted.
> I'm sending candidate patch.
>
> Martin
>
>
> 0001-Fix-params.exp-test-cases.patch
>
>
> From d5e3c5cb20306352af7f45cd827669b7ff51a709 Mon Sep 17 00:00:00 2001
> From: marxin <mliska@suse.cz>
> Date: Mon, 18 Jul 2016 09:50:12 +0200
> Subject: [PATCH] Fix params.exp test-cases
>
> gcc/testsuite/ChangeLog:
>
> 2016-07-18  Thomas Schwinge  <thomas@codesourcery.com>
> 	Martin Liska  <mliska@suse.cz>
>
> 	* gcc.dg/params/LICENSE: Add missing LICENSE that is mentioned
> 	in a source file.
> 	* gcc.dg/params/README: Same for the README.
> 	* gcc.dg/params/params.exp (param_run_test): Swap flags and
> 	default-extra-flags arguments when calling dg-runtest.
OK.

jeff