[libstdc++] testsuite tweaks, new makefile rules

[Phil Edwards <phil at jaj dot com>]

One of Bill Cosby's old comedy sketches has him telling his son, "I brought
you into this world, I'll take you out."  In the same vein, the Tcl code
that I hacked up to prepare a list of tests to run, which has been edited
and extended by many people, is hereby killed.

In its place is a separate shell script, more flexible and easier to
maintain.  The script is called by testsuite/Makefile as a precursor to
running the various suites.  This allows a maintainence programmer to
construct the list, edit it (the whole point of the list), and run the
suite without having to start dejagnu, count to five, then kill it[*], like

    % make testsuite_files_performance
    % $EDITOR testsuite_files
    % make check-performance

An unplanned change is that testsuite_files* are now built at the end of
"make all" rather than the beginning of "make check", in order to avoid
some suprising behavior with automake.  So, the first command above is
unneeded, assuming a normal build has already taken place.


Also some cosmetic formatting to abi_check help, and some cleanup of the
massive files created by the performance suite.  (I have ideas for automating
the cleanup of working files in general, but it's not really a problem yet.)

Tested on x86/linux, applied to trunk.


[*]  This patch largely inspired by expect's tendency to head off into
     east hyperspace and lock up into a loop on receiving a Ctrl-C.



2003-07-05  Phil Edwards  <pme@gcc.gnu.org>

	* scripts/create_testsuite_files:  New file.
	* testsuite/Makefile.am (all-local, check-performance):  Use it.
	* testsuite/lib/libstdc++-v3-dg.exp (v3-computer-tests):  Remove.
	* testsuite/Makefile.in:  Regenerated.

	* testsuite/performance/filebuf_sputc.cc:  Remove the temporary
	files at the end.
	* testsuite/performance/fstream_seek_write.cc:  Likewise.
	* testsuite/performance/ofstream_insert_float.cc:  Likewise.
	* testsuite/performance/ofstream_insert_int.cc:  Likewise.
	* testsuite/abi_check.cc (main):  Nicer spacing in usage output.


Index: scripts/create_testsuite_files
===================================================================
RCS file: scripts/create_testsuite_files
diff -N scripts/create_testsuite_files
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ scripts/create_testsuite_files	6 Jul 2003 03:11:59 -0000
@@ -0,0 +1,58 @@
+#!/bin/sh
+
+# Constructs lists of source files (full pathnames) to test.  Two
+# files are constructed: testsuite_files, which is used to test with
+# the default dg-runtest command, and testsuite_files_interactive,
+# which is used to test cases that require input to be entered. In
+# addition, both lists are pruned of wchar_t tests if the toolchain
+# under test does not support wchar_t functionality.
+#
+# We mimic the mkcheck script in that the first time this is run, all
+# existing files are listed in "testsuite_files" in the output
+# directory.  Subsequent runs pull the list from that file, allowing
+# users to trim the list down to problematic tests, or just run
+# paticular directories or sub-directories of tests.
+#
+# Selecting individual tests can also be done with RUNTESTFLAGS, but
+# that doesn't really do all that we are trying to accomplish here.
+
+LC_ALL=C
+export LC_ALL
+
+# Both of these are in the appropriate testsuite subdirectories.
+srcdir="$1"
+outdir="$2"
+
+tmp="${TMPDIR:-/tmp}/ctt$$"
+tests_file_normal="$outdir/testsuite_files"
+tests_file_inter="$outdir/testsuite_files_interactive"
+tests_file_perf="$outdir/testsuite_files_performance"
+
+cd $srcdir
+# This is the ugly version of "everything but the current directory".  It's
+# what has to happen when find(1) doesn't support -mindepth.
+dlist=`echo [0-9][0-9]*`
+for d in [a-z]*; do
+  test -d $d && dlist="$dlist $d"
+done
+find $dlist -type f -name "*.cc" | sort > $tmp.1
+
+# If the library is not configured to support wchar_t, don't run those tests.
+if test -f "$outdir/testsuite_wchar_t"; then
+  mv $tmp.1 $tmp.2
+else
+  grep -v wchar_t $tmp.1 > $tmp.2
+fi
+
+# Now filter out classes of tests.  These classes are run using special rules.
+grep _xin $tmp.2 > $tests_file_inter
+grep -v _xin $tmp.2 > $tmp.3
+
+grep performance $tmp.3 > $tests_file_perf
+grep -v performance $tmp.3 > $tmp.4
+
+# ...more filters go here.
+cp $tmp.4 $tests_file_normal
+
+rm $tmp*
+exit 0
Index: testsuite/Makefile.am
===================================================================
RCS file: /cvs/gcc/gcc/libstdc++-v3/testsuite/Makefile.am,v
retrieving revision 1.27
diff -u -3 -p -r1.27 Makefile.am
--- testsuite/Makefile.am	5 Jul 2003 04:05:43 -0000	1.27
+++ testsuite/Makefile.am	6 Jul 2003 03:11:59 -0000
@@ -38,6 +38,7 @@ RUNTEST = `if [ -f @glibcxx_srcdir@/../d
 AM_MAKEFLAGS = -j1
 AM_RUNTESTFLAGS =
 RUNTESTFLAGS =
+PWD_COMMAND = $${PWDCMD-pwd}
 
 ## CXX is actually a "C" compiler. These are real C++ programs.
 glibcxx_srcdir=@glibcxx_srcdir@
@@ -57,6 +58,13 @@ INCLUDES = \
 	-nostdinc++ \
 	@GLIBCXX_INCLUDES@ @LIBSUPCXX_INCLUDES@ @TOPLEVEL_INCLUDES@ 
 
+# Generated lists of files to run.  All of these names are valid make
+# targets, if you wish to generate a list manually.
+lists_of_files = \
+   testsuite_files \
+   testsuite_files_interactive \
+   testsuite_files_performance
+
 ## Build support library.
 noinst_LIBRARIES = libv3test.a
 libv3test_a_SOURCES = testsuite_hooks.cc testsuite_allocator.cc
@@ -69,7 +77,7 @@ noinst_PROGRAMS = 
 endif
 abi_check_SOURCES = abi_check.cc
 
-all-local: stamp_wchar
+all-local: stamp_wchar testsuite_files
 
 # Enable wchar_t tests if capable.
 if GLIBCXX_TEST_WCHAR_T
@@ -144,9 +152,16 @@ check-script-install: ${survey_script}
 # ammounts of memory, or otherwise tie up machine resources. Thus,
 # running this is off by default.
 performance_script=${glibcxx_srcdir}/scripts/check_performance
-check-performance: ${performance_script}
+check-performance: testsuite_files_performance ${performance_script}
 	-@(chmod + ${performance_script}; \
 	  ${performance_script} ${glibcxx_srcdir} ${glibcxx_builddir})
+
+
+# This rule generates all of the testsuite_files* lists at once.
+${lists_of_files}:
+	${glibcxx_srcdir}/scripts/create_testsuite_files \
+	  ${glibcxx_srcdir}/testsuite `${PWD_COMMAND}`
+
 
 .PHONY: baseline_symbols new-abi-baseline check-abi check-abi-verbose \
 check-script check-script-install check-performance
Index: testsuite/abi_check.cc
===================================================================
RCS file: /cvs/gcc/gcc/libstdc++-v3/testsuite/abi_check.cc,v
retrieving revision 1.18
diff -u -3 -p -r1.18 abi_check.cc
--- testsuite/abi_check.cc	5 Jul 2003 04:05:43 -0000	1.18
+++ testsuite/abi_check.cc	6 Jul 2003 03:11:59 -0000
@@ -317,7 +317,7 @@ main(int argc, char** argv)
   string argv1 = argc > 1 ? argv[1] : "";
   if (argv1 == "--help" || argc < 4)
     {
-      cerr << "usage: abi_check --check current baseline\n"
+      cerr << "usage: abi_check --check         current baseline\n"
               "                 --check-verbose current baseline\n"
               "                 --help\n\n"
               "Where CURRENT is a file containing the current results from\n"
Index: testsuite/lib/libstdc++-v3-dg.exp
===================================================================
RCS file: /cvs/gcc/gcc/libstdc++-v3/testsuite/lib/libstdc++-v3-dg.exp,v
retrieving revision 1.26
diff -u -3 -p -r1.26 libstdc++-v3-dg.exp
--- testsuite/lib/libstdc++-v3-dg.exp	30 Jun 2003 17:17:58 -0000	1.26
+++ testsuite/lib/libstdc++-v3-dg.exp	6 Jul 2003 03:12:00 -0000
@@ -59,9 +59,6 @@ proc libstdc++-v3-init { args } {
     libstdc++-v3-copy-files [glob -nocomplain "$srcdir/data/*.tst"] $outdir
     libstdc++-v3-copy-files [glob -nocomplain "$srcdir/data/*.txt"] $outdir
 
-    # Compute lists of files to test.
-    v3-compute-tests
-
     # set LD_LIBRARY_PATH so that libgcc_s, libstdc++ binaries can be found.
     # locate libgcc.a so we don't need to account for different values of
     # SHLIB_EXT on different platforms
@@ -217,115 +214,6 @@ proc libstdc++-v3_target_compile { sourc
     return [target_compile $source $dest $type $options]
 }
 
-
-# Constructs lists of source files (full pathnames) to test.  Two
-# files are constructed: testsuite_files, which is used to test with
-# the default dg-runtest command, and testsuite_files_interactive,
-# which is used to test cases that require input to be entered. In
-# addition, both lists are pruned of wchar_t tests if the toolchain
-# under test does not support wchar_t functionality.
-#
-# We mimic the mkcheck script in that the first time this is run, all
-# existing files are listed in "testsuite_files" in the output
-# directory.  Subsequent runs pull the list from that file, allowing
-# users to trim the list down to problematic tests, or just run
-# paticular directories or sub-directories of tests.
-#
-# Selecting individual tests can also be done with RUNTESTFLAGS, but
-# that doesn't really do all that we are trying to accomplish here.
-#
-# Essentially "static" to this file.
-proc v3-compute-tests { } {
-    global srcdir
-    global outdir
-
-    set tests_file "${outdir}/testsuite_files"
-    set tests_file_inter "${outdir}/testsuite_files_interactive"
-    set tests_file_perf "${outdir}/testsuite_files_performance"
-    set sfiles ""
-
-    # If there is a testsuite_file, use it. 
-    if { [file exists $tests_file] } {
-	return 
-    } 
-
-    # If not, generate it. 
-    set where_we_were [pwd]
-    cd $srcdir
-    foreach s [lsort [glob -nocomplain "*/*.cc" "*/*/*.cc" "{,*/}*/*/*/*.cc" ]] {
-	lappend sfiles ${s}
-    }
-    cd $where_we_were
-
-    # Filter wchar_t tests.
-    # Disable wchar_t tests if library not configured to support
-    # wchar_t testing. 
-    set wchar_file "${outdir}/testsuite_wchar_t"    
-    if { ! [file exists $wchar_file] } {
-	# Remove wchar_t tests files from list.
-	set res {}
-	foreach w $sfiles {
-	    if [regexp "wchar_t" $w] {
-		verbose "element wchar list is $w"
-	    } else {
-		verbose "element non-wchar list is $w"
-		lappend res $w
-	    }
-	}
-	set sfiles ${res}
-    }
-
-    # Filter interactive tests.
-    # Use special rules to run these tests.
-    set restwo {}
-    set sfiles_inter {}
-    foreach i $sfiles {
-	if [regexp "_xin" $i] {
-	    verbose "element interactive list is $i"
-	    lappend sfiles_inter $i
-	} else {
-	    verbose "element non-interactive list is $i"
-	    lappend restwo $i
-	}
-    }
-    set sfiles ${restwo}
-
-    # Filter performance analysis tests.
-    # Use special rules to run these tests.
-    set resthree {}
-    set sfiles_perf {}
-    foreach i $sfiles {
-	if [regexp "performance" $i] {
-	    verbose "element performance list is $i"
-	    lappend sfiles_perf $i
-	} else {
-	    verbose "element non-performancet list is $i"
-	    lappend resthree $i
-	}
-    }
-    set sfiles ${resthree}
-
-    # Write out testsuite_files.
-    set f [open $tests_file "w"]
-    foreach t $sfiles {
-	puts $f $t
-    }
-    close $f
-
-    # Write out testsuite_files_interactive.
-    set f [open $tests_file_inter "w"]
-    foreach t $sfiles_inter {
-	puts $f $t
-    }
-    close $f
-
-    # Write out testsuite_files_performance.
-    set f [open $tests_file_perf "w"]
-    foreach t $sfiles_perf {
-	puts $f $t
-    }
-    close $f
-}
 
 # Called once, from libstdc++-v3.dg/dg.exp.
 proc v3-list-tests { filename } {
Index: testsuite/performance/filebuf_sputc.cc
===================================================================
RCS file: /cvs/gcc/gcc/libstdc++-v3/testsuite/performance/filebuf_sputc.cc,v
retrieving revision 1.1
diff -u -3 -p -r1.1 filebuf_sputc.cc
--- testsuite/performance/filebuf_sputc.cc	19 Jun 2003 04:00:59 -0000	1.1
+++ testsuite/performance/filebuf_sputc.cc	6 Jul 2003 03:12:00 -0000
@@ -69,5 +69,6 @@ int main() 
   stop_counters(time, resource);
   report_performance(__FILE__, "C++", time, resource);
 
+  unlink("tmp");
   return 0;
 }
Index: testsuite/performance/fstream_seek_write.cc
===================================================================
RCS file: /cvs/gcc/gcc/libstdc++-v3/testsuite/performance/fstream_seek_write.cc,v
retrieving revision 1.2
diff -u -3 -p -r1.2 fstream_seek_write.cc
--- testsuite/performance/fstream_seek_write.cc	19 Jun 2003 04:00:59 -0000	1.2
+++ testsuite/performance/fstream_seek_write.cc	6 Jul 2003 03:12:00 -0000
@@ -53,5 +53,6 @@ int main() 
       report_performance(__FILE__, "", time, resource);
     }
 
+  unlink("tmp_perf_seek");
   return 0;
 }
Index: testsuite/performance/ofstream_insert_float.cc
===================================================================
RCS file: /cvs/gcc/gcc/libstdc++-v3/testsuite/performance/ofstream_insert_float.cc,v
retrieving revision 1.2
diff -u -3 -p -r1.2 ofstream_insert_float.cc
--- testsuite/performance/ofstream_insert_float.cc	19 Jun 2003 04:00:59 -0000	1.2
+++ testsuite/performance/ofstream_insert_float.cc	6 Jul 2003 03:12:00 -0000
@@ -48,5 +48,6 @@ int main() 
   stop_counters(time, resource);
   report_performance(__FILE__, "", time, resource);
 
+  unlink("tmp_perf_float.txt");
   return 0;
 };
Index: testsuite/performance/ofstream_insert_int.cc
===================================================================
RCS file: /cvs/gcc/gcc/libstdc++-v3/testsuite/performance/ofstream_insert_int.cc,v
retrieving revision 1.2
diff -u -3 -p -r1.2 ofstream_insert_int.cc
--- testsuite/performance/ofstream_insert_int.cc	19 Jun 2003 04:00:59 -0000	1.2
+++ testsuite/performance/ofstream_insert_int.cc	6 Jul 2003 03:12:00 -0000
@@ -45,5 +45,6 @@ int main() 
   stop_counters(time, resource);
   report_performance(__FILE__, "", time, resource);
 
+  unlink("tmp_perf_int.txt");
   return 0;
 };