This is the mail archive of the
gcc-patches@gcc.gnu.org
mailing list for the GCC project.
[PATCH] fixes to consistency.vlad/vlad.exp
- From: Janis Johnson <janis187 at us dot ibm dot com>
- To: gcc-patches at gcc dot gnu dot org
- Date: Fri, 23 Aug 2002 17:25:50 -0700
- Subject: [PATCH] fixes to consistency.vlad/vlad.exp
This patch allows the gcc/testsuite/consistency.vlad tests to run again.
They were apparently broken by changes to the test suite framework a few
months ago. I also changed the name of a variable that confused me,
made the target triple the default name of the baseline results, and
attempted to make the exp file more portable. C++ tests were built
using "gcc -x c++", which caused failures for tests with vtables, so now
they are built with g++ instead.
There are several issues yet to address, including:
- how to store baseline results; can they be stored as gzipped tarballs
to save space?
- how to identify baseline results, and figure out which ones to use in
the common case
Tested by simple builds of c,c++ followed by "make -k check-consistency"
to create baseline files which were copied to the expected location in
the test suite sources, then "make check-consistency" again to see that
tests passed. I also "broke" a couple of expected result files to check
that failures are handled appropriately. These steps were done on
i686-pc-linux-gnu and ia64-unknown-linux-gnu.
By the way, when I used "runtest tclvar=value" it complained about
"Illegal Argument", so the example in the README file showing how to
set variables via RUNTESTFLAGS doesn't work for me. I did try testing
with other flags by hacking the site.exp file.
Eventually I'd like to see these changes in the 3.2 branch, but for now
are they OK for the mainline?
2002-08-23 Janis Johnson <janis187@us.ibm.com>
* Makefile.in (check-consistency): Update to match recent changes to
other check targets.
2002-08-23 Janis Johnson <janis187@us.ibm.com>
* consistency.vlad/README: Update.
* consistency.vlad/vlad.exp: Update to work with recent changes to
test framework; rename BASE_COMPILER as CONSISTENCY_BASE; build
C++ executables with g++ driver; use built-in diff.
Index: Makefile.in
===================================================================
RCS file: /cvs/gcc/gcc/gcc/Makefile.in,v
retrieving revision 1.939
diff -u -p -r1.939 Makefile.in
--- Makefile.in 22 Aug 2002 04:29:34 -0000 1.939
+++ Makefile.in 23 Aug 2002 23:07:13 -0000
@@ -3210,15 +3210,15 @@ check-objc: $(TESTSUITEDIR)/site.exp
export TCL_LIBRARY ; fi ; \
$(RUNTEST) --tool objc $(RUNTESTFLAGS))
-check-consistency: testsuite/site.exp
- -rootme=`${PWD}`; export rootme; \
+check-consistency: $(TESTSUITEDIR)/site.exp
+ -(rootme=`${PWD}`; export rootme; \
srcdir=`cd ${srcdir}; ${PWD}` ; export srcdir ; \
- cd testsuite; \
+ cd $(TESTSUITEDIR); \
EXPECT=${EXPECT} ; export EXPECT ; \
if [ -f $${rootme}/../expect/expect ] ; then \
TCL_LIBRARY=`cd .. ; cd ${srcdir}/../tcl/library ; ${PWD}` ; \
export TCL_LIBRARY ; fi ; \
- $(RUNTEST) --tool consistency $(RUNTESTFLAGS)
+ $(RUNTEST) --tool consistency $(RUNTESTFLAGS))
# QMTest targets
Index: testsuite/consistency.vlad/README
===================================================================
RCS file: /cvs/gcc/gcc/gcc/testsuite/consistency.vlad/README,v
retrieving revision 1.1
diff -u -p -r1.1 README
--- testsuite/consistency.vlad/README 12 Feb 2001 19:39:54 -0000 1.1
+++ testsuite/consistency.vlad/README 23 Aug 2002 23:07:18 -0000
@@ -1,32 +1,32 @@
This directory contains tests and scripts for consistency checking
compilers. They could be used for consistency checking different
-C/C++ compiler for one platform or C/C++ compilers for different
-platforms.
+releases of the same C/C++ compiler, or different C/C++ compilers for
+the same platform, or C/C++ compilers for different platforms.
For example, to check the consistency, you could type
- make check-consistency RUNTESTFLAGS='BASE_COMPILER=i960-97r2 CONSISTENCY_OPTIONS="-O2 -fvtable-thunks -mno-strict-align -fno-rtti -fcheck-new -mcall-i960-old -mlittle-endian" NO_LONG_DOUBLE=1 NO_LIBGLOSS_LIBRARY_OPTIONS=1'
+ make check-consistency RUNTESTFLAGS='CONSISTENCY_BASE=i960-97r2 CONSISTENCY_OPTIONS="-O2 -fvtable-thunks -mno-strict-align -fno-rtti -fcheck-new -mcall-i960-old -mlittle-endian" NO_LONG_DOUBLE=1 NO_LIBGLOSS_LIBRARY_OPTIONS=1'
- In this example, DEJAGNU driver vlad.exp will process all C/C++
-tests in all subdirectories (e.g. layout) started with lower case
-letter. C tests (with suffixes .c) will be compiled by C compiler
-only. C++ tests will be compiled by C and C++ compilers. It is
+ In this example, the DEJAGNU driver vlad.exp will process all C/C++
+tests in all subdirectories (e.g. layout) that begin with a lower case
+letter. C++ tests (with suffixes .cpp) will be compiled by the C++
+compiler only. C tests will be compiled by C and C++ compilers. It is
supposed that each test outputs the result which is saved in temp_dir
as
<basename of the test>.out
or
<basename of the test>.out++
-correspondingly for C compiler and C++ compiler. For example, there
+correspondingly for the C compiler and C++ compiler. For example, there
will be two output files
`align-double.out' and `align-double.out++'
for test `align-double.c'.
- After this if there is subdirectory <BASE_COMPILER>-results
-(`i960-97r2-results' in our example) and there is corresponding saved
-output file for the base compiler the the files are compared in order
+ After this if there is a subdirectory <CONSISTENCY_BASE>-results
+(`i960-97r2-results' in our example) and there is a corresponding saved
+output file for the base compiler then the files are compared in order
to check consistency.
To create the base for future comparisons of output files, you
@@ -35,32 +35,33 @@ directory into the created directory.
The test script file has the following parameters
- 1. BASE_COMPILER. The variable must be defined for work the script
- properly.
+ 1. CONSISTENCY_BASE. The default is the target-triplet.
2. CONSISTENCY_OPTIONS. The options which will be used for
compilations. The default value is "-O2".
- 3. NO_LONG_DOUBLE. If the variable value is not zero, the tests
- which contain long double are expected to be fail.
+ 3. NO_LONG_DOUBLE. If the variable is defined and its value is not
+ zero, the tests which contain long double are expected to fail.
- 4. NO_LIBGLOSS_LIBRARY_OPTIONS. If the variable value is not zero,
- the tests are compiled without standard dejagnu libgloss library
- paths.
-
- Currently, there is directory `layout' for checking allocation
-parameters of C/C++ compiler and subdirectory i960-97r2 which contains
-the test results for 97r2 i960-vxworks C/C++ compiler.
+ 4. NO_LIBGLOSS_LIBRARY_OPTIONS. If the variable is defined and its
+ value is not zero, the tests are compiled without standard dejagnu
+ libgloss library paths.
+
+ Currently, there is a directory `layout' for checking the allocation
+parameters of a C/C++ compiler and a subdirectory i960-97r2-results
+which contains the test results for the 97r2 i960-vxworks C/C++
+compiler.
You can create other consistency tests. Simply, create a directory
-started with lower case letter, place the tests and create the
+starting with a lower case letter, place the tests and create the
subdirectory with the test results for a base compiler. If you need
to compile tests in a directory with a wrap file simply place the file
name (e.g. _dummy_stab.S) in file WRAP_FILE in the test directory.
The tests will be only compiled (and the test results will not be
compared) if you place <basename of the test>.conly in the directory
-of the test. You can use own driver for checking consistency. To
-make this, place the TCL script with name <basename of the test>.cexp.
+of the test. You can use your own driver for checking consistency. To
+make this, place the TCL script into a file with the name <basename of
+the test>.cexp.
Vladimir Makarov <vmakarov@cygnus.com>
Index: testsuite/consistency.vlad/vlad.exp
===================================================================
RCS file: /cvs/gcc/gcc/gcc/testsuite/consistency.vlad/vlad.exp,v
retrieving revision 1.2
diff -u -p -r1.2 vlad.exp
--- testsuite/consistency.vlad/vlad.exp 9 Oct 2001 10:16:15 -0000 1.2
+++ testsuite/consistency.vlad/vlad.exp 23 Aug 2002 23:09:42 -0000
@@ -1,4 +1,4 @@
-# Copyright (C) 1998,1999 Free Software Foundation, Inc.
+# Copyright (C) 1998,1999, 2002 Free Software Foundation, Inc.
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -20,18 +20,30 @@
# This file was created for by Vladimir Makarov (vmakarov@cygnus.com)
# on the base c-torture.exp.
-# The script requires setting up BASE_COMPILER (e.g. in RUNTESTFLAGS).
-# The default options can be overridden by
+# CONSISTENCY_BASE identifies the baseline data against which to compare the
+# results. By default this is the target triplet; it can be overridden
+# by setting CONSISTENCY_BASE (e.g. in RUNTESTFLAGS).
+#
+# The default compiler options can be overridden by
# CONSISTENCY_OPTIONS="option ... " (e.g. in RUNTESTFLAGS).
+# Load support procs.
+load_lib gcc.exp
+load_lib g++.exp
+
+# Initialize compiler-specific support.
+gcc_init
+g++_init
if $tracelevel then {
strace $tracelevel
}
-if ![info exists BASE_COMPILER] {
- fail "BASE_COMPILER is not set"
- return
+global CONSISTENCY_BASE CONSISTENCY_OPTIONS
+global NO_LONG_DOUBLE NO_LIBGLOSS_LIBRARY_OPTIONS
+
+if ![info exists CONSISTENCY_BASE] {
+ set CONSISTENCY_BASE $target_triplet
}
if ![info exists CONSISTENCY_OPTIONS] {
@@ -51,7 +63,7 @@ puts $CONSISTENCY_OPTIONS
regsub -- -.* $target_triplet "" target_chip
#
-# c-consistency-execute -- utility to compile and execute a testcase
+# consistency-execute -- utility to compile and execute a testcase
#
# SRC is the full pathname of the testcase.
#
@@ -61,10 +73,11 @@ regsub -- -.* $target_triplet "" target_
# run the compile.exp tests for example - one need only pass compile.exp to
# dejagnu, and not compile.exp, foo1.exp, foo2.exp, etc.).
#
-proc c-consistency-execute { src cpp_flag out_suffix} {
+proc consistency-execute { src comp_tool out_suffix} {
global tmpdir tool srcdir output
- global exec_output target_chip BASE_COMPILER target_triplet
- global CONSISTENCY_OPTIONS NO_LONG_DOUBLE
+ global exec_output target_chip target_triplet
+ global CONSISTENCY_BASE CONSISTENCY_OPTIONS
+ global NO_LONG_DOUBLE NO_LIBGLOSS_LIBRARY_OPTIONS
set consistency_options $CONSISTENCY_OPTIONS
@@ -111,28 +124,28 @@ proc c-consistency-execute { src cpp_fla
close $fd
}
- set comp_output [gcc_target_compile "$cpp_flag $src -x none $wrap_file" "$executable" executable $options];
+ set comp_output [${comp_tool}_target_compile "$src -x none $wrap_file" "$executable" executable $options];
# Set a few common compiler messages.
set fatal_signal "*cc: Internal compiler error: program*got fatal signal"
if [string match "$fatal_signal 6" $comp_output] then {
- fail $testcase "Got Signal 6, $cpp_flag $consistency_options"
+ fail $testcase "Got Signal 6, $comptool $consistency_options"
remote_file build delete $executable
return
}
if [string match "$fatal_signal 11" $comp_output] then {
- fail $testcase "Got Signal 11, $cpp_flag $consistency_options"
+ fail $testcase "Got Signal 11, $comp_tool $consistency_options"
remote_file build delete $executable
return
}
# We shouldn't get these because of -w, but just in case.
if [string match "*cc:*warning:*" $comp_output] then {
- warning "$testcase: (with warnings) $cpp_flag $consistency_options"
+ warning "$testcase: (with warnings) $comp_tool $consistency_options"
send_log "$comp_output\n"
- unresolved "$testcase, $cpp_flag $consistency_options"
+ unresolved "$testcase, $comp_tool $consistency_options"
remote_file build delete $executable
return
}
@@ -141,17 +154,17 @@ proc c-consistency-execute { src cpp_fla
if { ![file exists $executable] } {
if ![is3way] {
- fail "$testcase compilation, $cpp_flag $consistency_options"
- untested "$testcase execution, $cpp_flag $consistency_options"
+ fail "$testcase compilation, $comp_tool $consistency_options"
+ untested "$testcase execution, $comp_tool $consistency_options"
return
} else {
# FIXME: since we can't test for the existance of a remote
# file without short of doing an remote file list, we assume
# that since we got no output, it must have compiled.
- pass "$testcase compilation, $cpp_flag $consistency_options"
+ pass "$testcase compilation, $comp_tool $consistency_options"
}
} else {
- pass "$testcase compilation, $cpp_flag $consistency_options"
+ pass "$testcase compilation, $comp_tool $consistency_options"
}
# Check for compilation only.
@@ -190,7 +203,7 @@ proc c-consistency-execute { src cpp_fla
if { $status == "pass" } {
remote_file build delete $executable
}
- $status "$testcase execution, $cpp_flag $consistency_options"
+ $status "$testcase execution, $comp_tool $consistency_options"
if { $status != "pass" } {
return
@@ -204,7 +217,7 @@ proc c-consistency-execute { src cpp_fla
}
close $outf
- set base_outfile_name "$dirname/${BASE_COMPILER}-results/$basename.$out_suffix"
+ set base_outfile_name "$dirname/${CONSISTENCY_BASE}-results/$basename.$out_suffix"
# Check that the output for another processor has been saved
if { ![file exists $base_outfile_name] } {
@@ -224,11 +237,11 @@ proc c-consistency-execute { src cpp_fla
}
# Compare the outputs
- if {![catch "exec diff -b $outfile_name $base_outfile_name" message]} {
- pass "$testcase consistency, $cpp_flag $consistency_options"
+ if { [diff $outfile_name $base_outfile_name] == 1} then {
+ pass "$testcase consistency, $comp_tool $consistency_options"
+ remote_file build delete $outfile_name
} else {
- regsub "\n\[^>\n]\[^\n]*\$" $message "\n" difference
- fail "$testcase consistency, $cpp_flag $consistency_options\n$difference"
+ fail "$testcase consistency, $comp_tool $consistency_options"
}
}
@@ -265,11 +278,11 @@ foreach dir [lsort [glob -nocomplain $sr
if ![runtest_file_p $runtests $src] then {
continue
}
- regsub "^.*\\." $src "" suffix
+ regsub "^.*\\." $src "" suffix
if {$suffix != "cpp"} {
- c-consistency-execute $src "" "out"
+ consistency-execute $src "gcc" "out"
}
- c-consistency-execute $src "-x c++" "out++"
+ consistency-execute $src "g++" "out++"
}
}
}