Re: [wwwdocs] Initial criteria.html (was Re: Where's the current list of release criteria?)

[Gabriel Dos Reis <gdr at codesourcery dot com>]

<guerby@acm.org> writes:

| Joseph S. Myers wrote:
| <<It should use Dejagnu.>>
| 
| Could someone point me to the right files / directories to clone to
| create a simple Dejagnu harness for Ada? (Or even better, just create
| it in CVS with a simple Ada hello world program and I'll start from
| there :).

You might want to start with gcc/testsuite/g++.dg.
Since you're creating a new testsuite, you're encouraged to use the
new-style DejaGnu framework.


| To be able to fully use the executable part of ACATS and user
| executable bug reports, the test harness must be able to:
| 
| 1. Not run tests specially marked (eg: do not run test flagged
| "tasking" on a system where tasking is not supported, or test marked
| "kill" because they will kill your machine). The marking should be
| done either test by test in source or external file, or be based on
| list of tests (easier to manage).

If you just want to compile (and not exceute) a test, then your
testfile should contain something like (preferably at the beginning)

  -- { dg-do compile }

If you want to run it:

  -- { dg-do run }

If you just want a link

  -- { dg-do link }

| 2. Compile a set of support source files before processing a set of
| tests and be able to -I the place were support file were compiled
| when running those tests.

You can do that in a DejaGnu support script

| 3. Limit test compile and execution time.

This is not yet supported in DejaGnu -- we used a horrible hack in V3
land (Phil, no insult intended :-)

| 4. Compile some tests with special flags.

You can the dg-options keyword

  -- { dg-option "your-preferred-options" }

| 5. Match execution output with a given output, and be able to do some
| regexp matching on execution output to determine pass/failure status.

This functionality is something desirable in the V3 land but not yet
fully supported.  However, for specific cases you can have a good
approximate with "dg-final" feature.  I'm supposed to implement a
diff-ing support routine in the V3 land.

| 6. Compile each test starting in an empty directory.

You do that in the initialization script.

| 7. Run the whole test suite with a different set of flags.

You can do that as loop in ${tool}-runtest

| Examples on how to do the various points within Dejagnu for GCC are
| highly welcomed.

For looping you might want to look at gcc.torture (although we're
moving toward the new-style framework).  You might also want to do
some archeology in the V3 testsuite machinery and look into
libstc++-v3/testsuite before 2001-05-09.  

The DejaGnu manual *and* dejagnu/lib/dg.exp are the definitive source of
documentation. 

-- Gaby
CodeSourcery, LLC                       http://www.codesourcery.com