This is the mail archive of the gcc-patches@gcc.gnu.org mailing list for the GCC project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [RFC, v2] Test coverage for --param boundary values


On 07/21/2016 08:53 PM, Jeff Law wrote:
> On 07/18/2016 01:53 AM, Martin Liška wrote:
>> You are right, I was inspired by what we do for GCC plugins in:
>> gcc/testsuite/lib/plugin-support.exp
>>
>> where we have following comment:
>>     # Note that the plugin test support currently only works when the GCC
>>     # build tree is available. (We make sure that is the case in plugin.exp.)
>>     # Once we have figured out how/where to package/install GCC header files
>>     # for general plugin support, we should modify the following include paths
>>     # accordingly.
>>
>> Well, I can imaging a guard which will test whether "$objdir/../../params.options" file exits,
>> and if so, then the tests are executed? Is it acceptable approach?
> Yes.  That's acceptable to me.
> 
>>
>>> >
>>>> >> +set text [read $fd]
>>>> >> +close $fd
>>>> >> +
>>>> >> +# Main loop.
>>>> >> +foreach params [split $text "\n"] {
>>>> >> +    set parts [split $params "="]
>>>> >> +    set name [string trim [lindex $parts 0] '"']
>>>> >> +    set values [split [lindex $parts 1] ","]
>>>> >> +    if { [llength $values] == 3 } {
>>>> >> +    set default [lindex $values 0]
>>>> >> +    set min [lindex $values 1]
>>>> >> +    set max [lindex $values 2]
>>>> >> +    set int_max "INT_MAX"
>>>> >> +
>>>> >> +    if { $min != -1 } {
>>>> >> +        param_run_test $name $min
>>>> >> +    }
>>>> >> +    if { $max != $min && $max > 0 && $max != $int_max } {
>>>> >> +        param_run_test $name $max
>>>> >> +    }
>>>> >> +    }
>>>> >> +    if { [llength $values] == 5 } {
>>>> >> +    foreach v $values {
>>>> >> +        param_run_test $name $v
>>>> >> +    }
>>>> >> +    }
>>>> >> +}
>>> >
>>> > This produces ambiguous test result lines:
>>> >
>>> >     Running /scratch/tschwing/nvidiak20-2/gcc/trunk-light/source-gcc/gcc/testsuite/gcc.dg/params/params.exp ...
>>> >     PASS: gcc.dg/params/blocksort-part.c (test for excess errors)
>>> >     PASS: gcc.dg/params/blocksort-part.c (test for excess errors)
>>> >     PASS: gcc.dg/params/blocksort-part.c (test for excess errors)
>>> >     PASS: gcc.dg/params/blocksort-part.c (test for excess errors)
>>> >     [many more]
>>> >
>>> > OK to fix this as follows:
>>> >
>>> >     PASS: gcc.dg/params/blocksort-part.c -O3 --param predictable-branch-outcome=0 (test for excess errors)
>>> >     PASS: gcc.dg/params/blocksort-part.c -O3 --param predictable-branch-outcome=50 (test for excess errors)
>>> >     PASS: gcc.dg/params/blocksort-part.c -O3 --param inline-min-speedup=0 (test for excess errors)
>>> >     PASS: gcc.dg/params/blocksort-part.c -O3 --param max-inline-insns-single=0 (test for excess errors)
>>> >     [...]
>>> >
>>> > commit ae350ce9af062d7d2c05225280596b1bc982728b
>>> > Author: Thomas Schwinge <thomas@codesourcery.com>
>>> > Date:   Fri Jul 15 09:13:19 2016 +0200
>>> >
>>> >     Disambiguate gcc.dg/params/params.exp test results
>>> >
>>> >         gcc/testsuite/
>>> >         * gcc.dg/params/params.exp (param_run_test): Swap flags and
>>> >         default-extra-flags arguments when calling dg-runtest.
>>> > ---
>>> >  gcc/testsuite/gcc.dg/params/params.exp |    2 +-
>>> >  1 file changed, 1 insertion(+), 1 deletion(-)
>>> >
>>> > diff --git gcc/testsuite/gcc.dg/params/params.exp gcc/testsuite/gcc.dg/params/params.exp
>>> > index f2f397e..dd82a5d 100644
>>> > --- gcc/testsuite/gcc.dg/params/params.exp
>>> > +++ gcc/testsuite/gcc.dg/params/params.exp
>>> > @@ -28,7 +28,7 @@ proc param_run_test { param_name param_value } {
>>> >      global srcdir
>>> >      global subdir
>>> >
>>> > -    dg-runtest $srcdir/$subdir/blocksort-part.c "" "-O3 --param $param_name=$param_value"
>>> > +    dg-runtest $srcdir/$subdir/blocksort-part.c "-O3 --param $param_name=$param_value" ""
>>> >  }
>>> >
>>> >  set fd [open "$objdir/../../params.options" r]
>>> >
>>> >
>>> > Grüße
>>> >  Thomas
>>> >
>> Thanks for the patch, that's probably the same problem that Andreas spotted.
>> I'm sending candidate patch.
>>
>> Martin
>>
>>
>> 0001-Fix-params.exp-test-cases.patch
>>
>>
>> From d5e3c5cb20306352af7f45cd827669b7ff51a709 Mon Sep 17 00:00:00 2001
>> From: marxin <mliska@suse.cz>
>> Date: Mon, 18 Jul 2016 09:50:12 +0200
>> Subject: [PATCH] Fix params.exp test-cases
>>
>> gcc/testsuite/ChangeLog:
>>
>> 2016-07-18  Thomas Schwinge  <thomas@codesourcery.com>
>>     Martin Liska  <mliska@suse.cz>
>>
>>     * gcc.dg/params/LICENSE: Add missing LICENSE that is mentioned
>>     in a source file.
>>     * gcc.dg/params/README: Same for the README.
>>     * gcc.dg/params/params.exp (param_run_test): Swap flags and
>>     default-extra-flags arguments when calling dg-runtest.
> OK.
> 
> jeff
> 

Thanks.

I'm sending the version of patch I've just installed as r238633.

Martin
>From 920fce946873a1ec6a5a1755adc4a78fe4409f8e Mon Sep 17 00:00:00 2001
From: marxin <mliska@suse.cz>
Date: Mon, 18 Jul 2016 09:50:12 +0200
Subject: [PATCH] Fix params.exp test-cases

2016-07-18  Thomas Schwinge  <thomas@codesourcery.com>
	Martin Liska  <mliska@suse.cz>

	* gcc.dg/params/LICENSE: Add missing LICENSE that is mentioned
	in a source file.
	* gcc.dg/params/README: Same for the README.
	* gcc.dg/params/params.exp (param_run_test): Swap flags and
	default-extra-flags arguments when calling dg-runtest.
	* gcc.dg/params/params.exp: Run the test-cases only
	when params.options file exists.
---
 gcc/testsuite/gcc.dg/params/LICENSE    |  42 +++++++
 gcc/testsuite/gcc.dg/params/README     | 215 +++++++++++++++++++++++++++++++++
 gcc/testsuite/gcc.dg/params/params.exp |   9 +-
 3 files changed, 264 insertions(+), 2 deletions(-)
 create mode 100644 gcc/testsuite/gcc.dg/params/LICENSE
 create mode 100644 gcc/testsuite/gcc.dg/params/README

diff --git a/gcc/testsuite/gcc.dg/params/LICENSE b/gcc/testsuite/gcc.dg/params/LICENSE
new file mode 100644
index 0000000..cc61417
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/params/LICENSE
@@ -0,0 +1,42 @@
+
+--------------------------------------------------------------------------
+
+This program, "bzip2", the associated library "libbzip2", and all
+documentation, are copyright (C) 1996-2010 Julian R Seward.  All
+rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions
+are met:
+
+1. Redistributions of source code must retain the above copyright
+   notice, this list of conditions and the following disclaimer.
+
+2. The origin of this software must not be misrepresented; you must 
+   not claim that you wrote the original software.  If you use this 
+   software in a product, an acknowledgment in the product 
+   documentation would be appreciated but is not required.
+
+3. Altered source versions must be plainly marked as such, and must
+   not be misrepresented as being the original software.
+
+4. The name of the author may not be used to endorse or promote 
+   products derived from this software without specific prior written 
+   permission.
+
+THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
+OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
+DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
+GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
+WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+Julian Seward, jseward@bzip.org
+bzip2/libbzip2 version 1.0.6 of 6 September 2010
+
+--------------------------------------------------------------------------
diff --git a/gcc/testsuite/gcc.dg/params/README b/gcc/testsuite/gcc.dg/params/README
new file mode 100644
index 0000000..9fb0f63
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/params/README
@@ -0,0 +1,215 @@
+
+This is the README for bzip2/libzip2.
+This version is fully compatible with the previous public releases.
+
+------------------------------------------------------------------
+This file is part of bzip2/libbzip2, a program and library for
+lossless, block-sorting data compression.
+
+bzip2/libbzip2 version 1.0.6 of 6 September 2010
+Copyright (C) 1996-2010 Julian Seward <jseward@bzip.org>
+
+Please read the WARNING, DISCLAIMER and PATENTS sections in this file.
+
+This program is released under the terms of the license contained
+in the file LICENSE.
+------------------------------------------------------------------
+
+Complete documentation is available in Postscript form (manual.ps),
+PDF (manual.pdf) or html (manual.html).  A plain-text version of the
+manual page is available as bzip2.txt.
+
+
+HOW TO BUILD -- UNIX
+
+Type 'make'.  This builds the library libbz2.a and then the programs
+bzip2 and bzip2recover.  Six self-tests are run.  If the self-tests
+complete ok, carry on to installation:
+
+To install in /usr/local/bin, /usr/local/lib, /usr/local/man and
+/usr/local/include, type
+
+   make install
+
+To install somewhere else, eg, /xxx/yyy/{bin,lib,man,include}, type
+
+   make install PREFIX=/xxx/yyy
+
+If you are (justifiably) paranoid and want to see what 'make install'
+is going to do, you can first do
+
+   make -n install                      or
+   make -n install PREFIX=/xxx/yyy      respectively.
+
+The -n instructs make to show the commands it would execute, but not
+actually execute them.
+
+
+HOW TO BUILD -- UNIX, shared library libbz2.so.
+
+Do 'make -f Makefile-libbz2_so'.  This Makefile seems to work for
+Linux-ELF (RedHat 7.2 on an x86 box), with gcc.  I make no claims
+that it works for any other platform, though I suspect it probably
+will work for most platforms employing both ELF and gcc.
+
+bzip2-shared, a client of the shared library, is also built, but not
+self-tested.  So I suggest you also build using the normal Makefile,
+since that conducts a self-test.  A second reason to prefer the
+version statically linked to the library is that, on x86 platforms,
+building shared objects makes a valuable register (%ebx) unavailable
+to gcc, resulting in a slowdown of 10%-20%, at least for bzip2.
+
+Important note for people upgrading .so's from 0.9.0/0.9.5 to version
+1.0.X.  All the functions in the library have been renamed, from (eg)
+bzCompress to BZ2_bzCompress, to avoid namespace pollution.
+Unfortunately this means that the libbz2.so created by
+Makefile-libbz2_so will not work with any program which used an older
+version of the library.  I do encourage library clients to make the
+effort to upgrade to use version 1.0, since it is both faster and more
+robust than previous versions.
+
+
+HOW TO BUILD -- Windows 95, NT, DOS, Mac, etc.
+
+It's difficult for me to support compilation on all these platforms.
+My approach is to collect binaries for these platforms, and put them
+on the master web site (http://www.bzip.org).  Look there.  However
+(FWIW), bzip2-1.0.X is very standard ANSI C and should compile
+unmodified with MS Visual C.  If you have difficulties building, you
+might want to read README.COMPILATION.PROBLEMS.
+
+At least using MS Visual C++ 6, you can build from the unmodified
+sources by issuing, in a command shell: 
+
+   nmake -f makefile.msc
+
+(you may need to first run the MSVC-provided script VCVARS32.BAT
+ so as to set up paths to the MSVC tools correctly).
+
+
+VALIDATION
+
+Correct operation, in the sense that a compressed file can always be
+decompressed to reproduce the original, is obviously of paramount
+importance.  To validate bzip2, I used a modified version of Mark
+Nelson's churn program.  Churn is an automated test driver which
+recursively traverses a directory structure, using bzip2 to compress
+and then decompress each file it encounters, and checking that the
+decompressed data is the same as the original.
+
+
+
+Please read and be aware of the following:
+
+WARNING:
+
+   This program and library (attempts to) compress data by 
+   performing several non-trivial transformations on it.  
+   Unless you are 100% familiar with *all* the algorithms 
+   contained herein, and with the consequences of modifying them, 
+   you should NOT meddle with the compression or decompression 
+   machinery.  Incorrect changes can and very likely *will* 
+   lead to disastrous loss of data.
+
+
+DISCLAIMER:
+
+   I TAKE NO RESPONSIBILITY FOR ANY LOSS OF DATA ARISING FROM THE
+   USE OF THIS PROGRAM/LIBRARY, HOWSOEVER CAUSED.
+
+   Every compression of a file implies an assumption that the
+   compressed file can be decompressed to reproduce the original.
+   Great efforts in design, coding and testing have been made to
+   ensure that this program works correctly.  However, the complexity
+   of the algorithms, and, in particular, the presence of various
+   special cases in the code which occur with very low but non-zero
+   probability make it impossible to rule out the possibility of bugs
+   remaining in the program.  DO NOT COMPRESS ANY DATA WITH THIS
+   PROGRAM UNLESS YOU ARE PREPARED TO ACCEPT THE POSSIBILITY, HOWEVER
+   SMALL, THAT THE DATA WILL NOT BE RECOVERABLE.
+
+   That is not to say this program is inherently unreliable.  
+   Indeed, I very much hope the opposite is true.  bzip2/libbzip2 
+   has been carefully constructed and extensively tested.
+
+
+PATENTS:
+
+   To the best of my knowledge, bzip2/libbzip2 does not use any 
+   patented algorithms.  However, I do not have the resources 
+   to carry out a patent search.  Therefore I cannot give any 
+   guarantee of the above statement.
+
+
+
+WHAT'S NEW IN 0.9.0 (as compared to 0.1pl2) ?
+
+   * Approx 10% faster compression, 30% faster decompression
+   * -t (test mode) is a lot quicker
+   * Can decompress concatenated compressed files
+   * Programming interface, so programs can directly read/write .bz2 files
+   * Less restrictive (BSD-style) licensing
+   * Flag handling more compatible with GNU gzip
+   * Much more documentation, i.e., a proper user manual
+   * Hopefully, improved portability (at least of the library)
+
+WHAT'S NEW IN 0.9.5 ?
+
+   * Compression speed is much less sensitive to the input
+     data than in previous versions.  Specifically, the very
+     slow performance caused by repetitive data is fixed.
+   * Many small improvements in file and flag handling.
+   * A Y2K statement.
+
+WHAT'S NEW IN 1.0.0 ?
+
+   See the CHANGES file.
+
+WHAT'S NEW IN 1.0.2 ?
+
+   See the CHANGES file.
+
+WHAT'S NEW IN 1.0.3 ?
+
+   See the CHANGES file.
+
+WHAT'S NEW IN 1.0.4 ?
+
+   See the CHANGES file.
+
+WHAT'S NEW IN 1.0.5 ?
+
+   See the CHANGES file.
+
+WHAT'S NEW IN 1.0.6 ?
+
+   See the CHANGES file.
+
+
+I hope you find bzip2 useful.  Feel free to contact me at
+   jseward@bzip.org
+if you have any suggestions or queries.  Many people mailed me with
+comments, suggestions and patches after the releases of bzip-0.15,
+bzip-0.21, and bzip2 versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1,
+1.0.2 and 1.0.3, and the changes in bzip2 are largely a result of this
+feedback.  I thank you for your comments.
+
+bzip2's "home" is http://www.bzip.org/
+
+Julian Seward
+jseward@bzip.org
+Cambridge, UK.
+
+18     July 1996 (version 0.15)
+25   August 1996 (version 0.21)
+ 7   August 1997 (bzip2, version 0.1)
+29   August 1997 (bzip2, version 0.1pl2)
+23   August 1998 (bzip2, version 0.9.0)
+ 8     June 1999 (bzip2, version 0.9.5)
+ 4     Sept 1999 (bzip2, version 0.9.5d)
+ 5      May 2000 (bzip2, version 1.0pre8)
+30 December 2001 (bzip2, version 1.0.2pre1)
+15 February 2005 (bzip2, version 1.0.3)
+20 December 2006 (bzip2, version 1.0.4)
+10 December 2007 (bzip2, version 1.0.5)
+ 6     Sept 2010 (bzip2, version 1.0.6)
diff --git a/gcc/testsuite/gcc.dg/params/params.exp b/gcc/testsuite/gcc.dg/params/params.exp
index f2f397e..47f5af5 100644
--- a/gcc/testsuite/gcc.dg/params/params.exp
+++ b/gcc/testsuite/gcc.dg/params/params.exp
@@ -28,10 +28,15 @@ proc param_run_test { param_name param_value } {
     global srcdir
     global subdir
 
-    dg-runtest $srcdir/$subdir/blocksort-part.c "" "-O3 --param $param_name=$param_value"
+    dg-runtest $srcdir/$subdir/blocksort-part.c "-O3 --param $param_name=$param_value" ""
 }
 
-set fd [open "$objdir/../../params.options" r]
+set options_file "$objdir/../../params.options"
+if { [file exists $options_file] == 0 } {
+  return
+}
+
+set fd [open $options_file r]
 set text [read $fd]
 close $fd
 
-- 
2.9.0


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]