This is the mail archive of the gcc-patches@gcc.gnu.org mailing list for the GCC project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

[doc] Fixes to testsuite chapter in sourcebuild.texi


After Janis' hint, I've just completed reading the testsuite chapter in
the internals manual.  Overall, it's been very helpful, with one notable
exception (which is mentioned in some FIXME): there's no guide for
testsuite writers, something like an `Anatomy of a DejaGnu/GCC
Testsuite'.  I don't think the DejaGnu docs provide anything like this,
but would like to be proven wrong.

As a first experiment, I'd like to turn the single boehm-gc test into a
dg testsuite, since I've often noticed that failures there are the root
cause for libjava failures, but the failure goes completely unnoticed in
make check output.  Afterwards, I'll try to convert ACATS into a dg
testsuite as well, which will be more challening, but nicely resolves the
current lack of multilib testing.

Anyway, below's the first round of fixes to problems I've found, mostly
markup fixes and changing filenames for .exp files into pathnames
relative to gcc/testsuite.  Besides, I've completed (or even created)
keyword lists for dg-require-effective-target, dg-add-options,
dg-require-<support>, and dg-final.  Mostly, the lists are just
placeholders since I don't currently have the time to fill them in, but it
serves as a reminder what's missing.

The changes pass make doc/gccint.info and make doc/gccint.dvi, so at
least syntactically they're ok.  There's a strange problem with
formatting of the dvi output, though: in the table of effective-target
keywords, sometimes (at irregular intervals and seemingly unrelated to
the input) there's extra spacing between lines.

Ok for mainline?

	Rainer

-- 
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University


2010-03-15  Rainer Orth  <ro@CeBiTec.Uni-Bielefeld.DE>

	* doc/sourcebuild.texi (Testsuites): Fix markup.
	Use pathnames relative to gcc/testsuite.
	List effective-target keywords.
	(Test Directives): Add missing dg-add-options features.
	Move dg-timeout	description up.
	List dg-require-<support> variants.
	Sort list of dg-final local-directives, add placeholders for
	missing entries.
	(Ada Tests): Favor gnat.dg over ada/acats/tests/gcc.
	(C Tests): Correct gcc.misc-tests directory.
	Framework tests live in	gcc.test-framework now.

diff -r 88c968a26db9 -r e5ae183f9a76 gcc/doc/sourcebuild.texi
--- a/gcc/doc/sourcebuild.texi	Mon Mar 15 07:46:33 2010 +0100
+++ b/gcc/doc/sourcebuild.texi	Mon Mar 15 19:13:49 2010 +0100
@@ -1,4 +1,4 @@
-@c Copyright (C) 2002, 2003, 2004, 2005, 2007, 2008, 2009
+@c Copyright (C) 2002, 2003, 2004, 2005, 2007, 2008, 2009, 2010
 @c Free Software Foundation, Inc.
 @c This is part of the GCC manual.
 @c For copying conditions, see the file gcc.texi.
@@ -930,9 +930,171 @@
 context, the selector specifies whether a test is skipped and reported
 as unsupported or is expected to fail.  Use @samp{*-*-*} to match any
 target.
-Effective-target keywords are defined in @file{target-supports.exp} in
+Effective-target keywords are defined in @file{lib/target-supports.exp} in
 the GCC testsuite.
 
+FIXME: explain, move down to @code{dg-require-effective-target}?
+
+@table @code
+
+@item aes
+
+@item arm32
+
+@item arm_eabi
+
+@item arm_hard_vfp_ok
+
+@item arm_iwmmxt_ok
+
+@item arm_neon_hw
+
+@item arm_neon_ok
+
+@item arm_thumb1_ok
+
+@item arm_thumb2_ok
+
+@item arm_vfp_ok
+
+@item avx
+
+@item c99_runtime
+
+@item cxa_atexit
+@c @item "cxa_atexit"
+@c with and without quotes?
+
+@item dfp
+
+@item double64plus
+
+@item ealib
+@c @item "ealib"
+@c with quotes?
+
+@item fma4
+
+@item fortran_integer_16
+
+@item fortran_large_int
+
+@item fortran_large_real
+
+@item fpic
+
+@item freorder
+
+@item fstack_protector
+
+@item hard_float
+
+@item ilp32
+
+@item int32plus
+
+@item large_double
+
+@item lp64
+
+@item lto
+
+@item mips_newabi_large_long_double
+
+@item ms_hook_prologue
+
+@item named_sections
+
+@item nonpic
+
+@item pclmul
+
+@item powerpc64
+
+@item powerpc_altivec_ok
+
+@item powerpc_ppu_ok
+
+@item powerpc_spe
+
+@item powerpc_spu
+
+@item powerpc_vsx_ok
+
+@item pthread
+
+@item pthread_h
+
+@item ptr32plus
+
+@item section_anchors
+
+@item size32plus
+
+@item sse4
+
+@item sse4a
+
+@item ssse3
+
+@item static
+
+@item static_libgfortran
+
+@item stdint_types
+
+@item string_merging
+
+@item sync_char_short
+
+@item sync_int_long
+
+@item tls
+
+@item tls_native
+
+@item tls_runtime
+
+@item tls_runtime 
+
+@item trampolines
+
+@item ultrasparc_hw
+
+@item unwrapped
+
+@item vaes
+
+@item vect_condition
+
+@item vect_double
+
+@item vect_float
+
+@item vect_floatint_cvt
+
+@item vect_int
+
+@item vect_int_mult
+
+@item vect_intfloat_cvt
+
+@item vect_long
+
+@item vect_long_long
+
+@item vect_shift
+
+@item vect_uintfloat_cvt
+
+@item vpclmul
+
+@item wchar
+
+@item xop
+
+@end table
+
 A selector expression appears within curly braces and uses a single
 logical operator: one of @samp{!}, @samp{&&}, or @samp{||}.  An
 operand is another selector expression, an effective-target keyword,
@@ -972,10 +1134,10 @@
 then the test is skipped unless the target system is included in the
 list of target triplets or matches the effective-target keyword.
 
-If @samp{do-what-keyword} is @code{run} and the directive includes
+If @var{do-what-keyword} is @code{run} and the directive includes
 the optional @samp{@{ xfail @var{selector} @}} and the selector is met
 then the test is expected to fail.  The @code{xfail} clause is ignored
-for other values of @samp{do-what-keyword}; those tests can use
+for other values of @var{do-what-keyword}; those tests can use
 directive @code{dg-xfail-if}.
 
 @item @{ dg-options @var{options} [@{ target @var{selector} @}] @}
@@ -991,8 +1153,17 @@
 
 The supported values of @var{feature} are:
 @table @code
+@item arm_neon
+FIXME: @code{add_options_for_arm_neon} undefined.
+
+@item bind_pic_locally
+This test requires functions to bind locally when using pic/PIC passes.
+
 @item c99_runtime
-The target's C99 runtime (both headers and libraries).
+This test requires a C99 runtime (both headers and libraries).
+
+@item ieee
+This test requires full IEEE compliance mode.
 
 @item mips16_attribute
 @code{mips16} function attributes.  Only MIPS targets support this feature,
@@ -1001,11 +1172,7 @@
 
 @item @{ dg-timeout @var{n} [@{target @var{selector} @}] @}
 Set the time limit for the compilation and for the execution of the test
-to the specified number of seconds.
-
-@item @{ dg-timeout-factor @var{x} [@{ target @var{selector} @}] @}
-Multiply the normal time limit for compilation and execution of the test
-by the specified floating-point factor.  The normal timeout limit, in
+to the specified number of seconds. The normal timeout limit, in
 seconds, is found by searching the following in order:
 
 @itemize @bullet
@@ -1019,6 +1186,10 @@
 @item 300
 @end itemize
 
+@item @{ dg-timeout-factor @var{x} [@{ target @var{selector} @}] @}
+Multiply the normal time limit for compilation and execution of the test
+by the specified floating-point factor. 
+
 @item @{ dg-skip-if @var{comment} @{ @var{selector} @} [@{ @var{include-opts} @} [@{ @var{exclude-opts} @}]] @}
 Arguments @var{include-opts} and @var{exclude-opts} are lists in which
 each element is a string of zero or more GCC options.
@@ -1075,11 +1246,48 @@
 
 @item  @{ dg-xfail-run-if @var{comment} @{ @var{selector} @} [@{ @var{include-opts} @} [@{ @var{exclude-opts} @}]] @}
 Expect the execute step of a test to fail if the conditions (which are
-the same as for @code{dg-skip-if}) and @code{dg-xfail-if}) are met.
+the same as for @code{dg-skip-if} and @code{dg-xfail-if}) are met.
 
 @item @{ dg-require-@var{support} args @}
 Skip the test if the target does not provide the required support;
-see @file{gcc-dg.exp} in the GCC testsuite for the actual directives.
+see @file{lib/gcc-dg.exp} in the GCC testsuite for the actual directives.
+
+FIXME: explain?
+
+@table @code
+
+@c @item dg-require-alias
+@c with and without arg?
+@item dg-require-alias ""
+
+@item dg-require-compat-dfp ""
+
+@item dg-require-dll ""
+
+@item dg-require-false ""
+
+@item dg-require-gc-sections ""
+
+@item dg-require-host-local ""
+
+@item dg-require-iconv "@var{IBM1047}"
+
+@item dg-require-named-sections ""
+
+@item dg-require-profiling "@option{-fprofile-arcs}"
+
+@item dg-require-true ""
+
+@item dg-require-visibility "@var{internal}"
+
+@c @item dg-require-weak
+@c with and without arg?
+@item dg-require-weak ""
+
+@item dg-require-weak-override ""
+
+@end table
+
 These directives must appear after any @code{dg-do} directive in the test
 and before any @code{dg-additional-sources} directive.
 They require at least one argument, which can be an empty string if the
@@ -1088,6 +1296,7 @@
 @item @{ dg-require-effective-target @var{keyword} @}
 Skip the test if the test target, including current multilib flags,
 is not covered by the effective-target keyword.
+FIXME: explain difference to @code{dg-require-@var{support}}.
 This directive must appear after any @code{dg-do} directive in the test
 and before any @code{dg-additional-sources} directive.
 
@@ -1158,18 +1367,58 @@
 @item cleanup-coverage-files
 Removes coverage data files generated for this test.
 
+@item cleanup-ipa-dump @var{}
+
+@item cleanup-modules @var{}
+
+@item cleanup-profile-file
+
 @item cleanup-repo-files
 Removes files generated for this test for @option{-frepo}.
 
 @item cleanup-rtl-dump @var{suffix}
 Removes RTL dump files generated for this test.
 
+@item cleanup-saved-temps @var{}
+Removes files for the current test which were kept for @option{--save-temps}.
+
 @item cleanup-tree-dump @var{suffix}
 Removes tree dump files matching @var{suffix} which were generated for
 this test.
 
-@item cleanup-saved-temps
-Removes files for the current test which were kept for @option{--save-temps}.
+@item fail @var{}
+
+@item output-exists [@{ target/xfail @var{selector} @}]
+Passes if compiler output file exists.
+
+@item output-exists-not [@{ target/xfail @var{selector} @}]
+Passes if compiler output file does not exist.
+
+@item pass @var{}
+
+@item run-gcov @var{sourcefile}
+Check line counts in @command{gcov} tests.
+
+@item run-gcov [branches] [calls] @{ @var{opts} @var{sourcefile} @}
+Check branch and/or call counts, in addition to line counts, in
+@command{gcov} tests.
+
+@item scan-assembler @var{regex} [@{ target/xfail @var{selector} @}]
+Passes if @var{regex} matches text in the test's assembler output.
+
+@item scan-assembler-dem @var{regex} [@{ target/xfail @var{selector} @}]
+Passes if @var{regex} matches text in the test's demangled assembler output.
+
+@item scan-assembler-dem-not @var{regex} [@{ target/xfail @var{selector} @}]
+Passes if @var{regex} does not match text in the test's demangled assembler
+output.
+
+@item scan-assembler-not @var{regex} [@{ target/xfail @var{selector} @}]
+Passes if @var{regex} does not match text in the test's assembler output.
+
+@item scan-assembler-times @var{regex} @var{num} [@{ target/xfail @var{selector} @}]
+Passes if @var{regex} is matched exactly @var{num} times in the test's
+assembler output.
 
 @item scan-file @var{filename} @var{regexp} [@{ target/xfail @var{selector} @}]
 Passes if @var{regexp} matches text in @var{filename}.
@@ -1181,38 +1430,27 @@
 Passes if @var{symbol} is defined as a hidden symbol in the test's
 assembly output.
 
+@item scan-ipa-dump @var{} @var{}
+
+@item scan-ipa-dump-not @var{} @var{}
+
+@item scan-module @var{} @var{}
+
 @item scan-not-hidden @var{symbol} [@{ target/xfail @var{selector} @}]
 Passes if @var{symbol} is not defined as a hidden symbol in the test's
 assembly output.
 
-@item scan-assembler-times @var{regex} @var{num} [@{ target/xfail @var{selector} @}]
-Passes if @var{regex} is matched exactly @var{num} times in the test's
-assembler output.
+@item scan-rtl-dump @var{} @var{}
 
-@item scan-assembler @var{regex} [@{ target/xfail @var{selector} @}]
-Passes if @var{regex} matches text in the test's assembler output.
+@item scan-rtl-dump-not @var{} @var{}
 
-@item scan-assembler-not @var{regex} [@{ target/xfail @var{selector} @}]
-Passes if @var{regex} does not match text in the test's assembler output.
+@item scan-rtl-dump-times @var{} @var{}
 
-@item scan-assembler-dem @var{regex} [@{ target/xfail @var{selector} @}]
-Passes if @var{regex} matches text in the test's demangled assembler output.
-
-@item scan-assembler-dem-not @var{regex} [@{ target/xfail @var{selector} @}]
-Passes if @var{regex} does not match text in the test's demangled assembler
-output.
-
-@item scan-tree-dump-times @var{regex} @var{num} @var{suffix} [@{ target/xfail @var{selector} @}]
-Passes if @var{regex} is found exactly @var{num} times in the dump file
-with suffix @var{suffix}.
+@item scan-symbol @var{}
 
 @item scan-tree-dump @var{regex} @var{suffix} [@{ target/xfail @var{selector} @}]
 Passes if @var{regex} matches text in the dump file with suffix @var{suffix}.
 
-@item scan-tree-dump-not @var{regex} @var{suffix} [@{ target/xfail @var{selector} @}]
-Passes if @var{regex} does not match text in the dump file with suffix
-@var{suffix}.
-
 @item scan-tree-dump-dem @var{regex} @var{suffix} [@{ target/xfail @var{selector} @}]
 Passes if @var{regex} matches demangled text in the dump file with
 suffix @var{suffix}.
@@ -1221,27 +1459,26 @@
 Passes if @var{regex} does not match demangled text in the dump file with
 suffix @var{suffix}.
 
-@item output-exists [@{ target/xfail @var{selector} @}]
-Passes if compiler output file exists.
+@item scan-tree-dump-not @var{regex} @var{suffix} [@{ target/xfail @var{selector} @}]
+Passes if @var{regex} does not match text in the dump file with suffix
+@var{suffix}.
 
-@item output-exists-not [@{ target/xfail @var{selector} @}]
-Passes if compiler output file does not exist.
+@item scan-tree-dump-times @var{regex} @var{num} @var{suffix} [@{ target/xfail @var{selector} @}]
+Passes if @var{regex} is found exactly @var{num} times in the dump file
+with suffix @var{suffix}.
 
-@item run-gcov @var{sourcefile}
-Check line counts in @command{gcov} tests.
+@end table
 
-@item run-gcov [branches] [calls] @{ @var{opts} @var{sourcefile} @}
-Check branch and/or call counts, in addition to line counts, in
-@command{gcov} tests.
 @end table
-@end table
+
+FIXME: explain @code{dg-final-generate}, @code{dg-final-use}.
 
 @node Ada Tests
 @section Ada Language Testsuites
 
 The Ada testsuite includes executable tests from the ACATS 2.5
 testsuite, publicly available at
-@uref{http://www.adaic.org/compilers/acats/2.5}
+@uref{http://www.adaic.org/compilers/acats/2.5}.
 
 These tests are integrated in the GCC testsuite in the
 @file{gcc/testsuite/ada/acats} directory, and
@@ -1257,11 +1494,12 @@
 @end smallexample
 
 The tests are organized by directory, each directory corresponding to
-a chapter of the Ada Reference Manual.  So for example, c9 corresponds
+a chapter of the Ada Reference Manual.  So for example, @file{c9} corresponds
 to chapter 9, which deals with tasking features of the language.
 
 There is also an extra chapter called @file{gcc} containing a template for
-creating new executable tests.
+creating new executable tests, although this is deprecated in favor of
+the @file{gnat.dg} testsuite.
 
 The tests are run using two @command{sh} scripts: @file{run_acats} and
 @file{run_all.sh}.  To run the tests using a simulator or a cross
@@ -1291,7 +1529,7 @@
 are not run with multiple optimization options.
 @item gcc.dg/compat
 This subdirectory contains tests for binary compatibility using
-@file{compat.exp}, which in turn uses the language-independent support
+@file{lib/compat.exp}, which in turn uses the language-independent support
 (@pxref{compat Testing, , Support for testing binary compatibility}).
 @item gcc.dg/cpp
 This subdirectory contains tests of the preprocessor.
@@ -1341,21 +1579,19 @@
 FIXME: describe this.
 
 This directory should probably not be used for new tests.
-@item gcc.c-torture/misc-tests
+@item gcc.misc-tests
 This directory contains C tests that require special handling.  Some
 of these tests have individual expect files, and others share
 special-purpose expect files:
 
 @table @file
 @item @code{bprob*.c}
-Test @option{-fbranch-probabilities} using @file{bprob.exp}, which
+Test @option{-fbranch-probabilities} using
+@file{gcc.misc-tests/bprob.exp}, which
 in turn uses the generic, language-independent framework
 (@pxref{profopt Testing, , Support for testing profile-directed
 optimizations}).
 
-@item @code{dg-*.c}
-Test the testsuite itself using @file{dg-test.exp}.
-
 @item @code{gcov*.c}
 Test @command{gcov} output using @file{gcov.exp}, which in turn uses the
 language-independent support (@pxref{gcov Testing, , Support for testing gcov}).
@@ -1364,6 +1600,12 @@
 Test i386-specific support for data prefetch using @file{i386-prefetch.exp}.
 @end table
 
+@item gcc.test-framework
+@table @file
+@item @code{dg-*.c}
+Test the testsuite itself using @file{gcc.test-framework/test-framework.exp}.
+@end table
+
 @end table
 
 FIXME: merge in @file{testsuite/README.gcc} and discuss the format of
@@ -1376,10 +1618,10 @@
 @file{@var{target}/libjava/testsuite} directory in the build
 tree.  Additional runtime tests can be checked into this testsuite.
 
-Regression testing of the core packages in libgcj is also covered by the
+Regression testing of the core packages in @code{libgcj} is also covered by the
 Mauve testsuite.  The @uref{http://sourceware.org/mauve/,,Mauve Project}
 develops tests for the Java Class Libraries.  These tests are run as part
-of libgcj testing by placing the Mauve tree within the libjava testsuite
+of @code{libgcj} testing by placing the Mauve tree within the libjava testsuite
 sources at @file{libjava/testsuite/libjava.mauve/mauve}, or by specifying
 the location of that tree when invoking @samp{make}, as in
 @samp{make MAUVEDIR=~/mauve check}.
@@ -1388,7 +1630,7 @@
 failures for a test run against the list of expected failures in
 @file{libjava/testsuite/libjava.mauve/xfails} from the source hierarchy.
 Update this file when adding new failing tests to Mauve, or when fixing
-bugs in libgcj that had caused Mauve test failures.
+bugs in @code{libgcj} that had caused Mauve test failures.
 
 We encourage developers to contribute test cases to Mauve.
 
@@ -1434,7 +1676,7 @@
 Language-independent support for testing @command{gcov}, and for checking
 that branch profiling produces expected values, is provided by the
 expect file @file{gcov.exp}.  @command{gcov} tests also rely on procedures
-in @file{gcc.dg.exp} to compile and run the test program.  A typical
+in @file{gcc-dg.exp} to compile and run the test program.  A typical
 @command{gcov} test contains the following DejaGnu commands within comments:
 
 @smallexample
@@ -1607,7 +1849,7 @@
 rm site.exp
 make -k \
   ALT_CXX_UNDER_TEST=$@{alt_prefix@}/bin/g++ \
-  COMPAT_OPTIONS="lists as shown above" \
+  COMPAT_OPTIONS="@var{lists as shown above}" \
   check-c++ \
   RUNTESTFLAGS="compat.exp"
 @end smallexample
@@ -1646,7 +1888,7 @@
 Throughout the compiler testsuite there are several directories whose
 tests are run multiple times, each with a different set of options.
 These are known as torture tests.
-@file{gcc/testsuite/lib/torture-options.exp} defines procedures to
+@file{lib/torture-options.exp} defines procedures to
 set up these lists:
 
 @table @code


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]