Beginner's Guide to Writing Testcases

Give your testcases an unequivocal name, ending with an underscore and a number: line_length_1.f and line_length_1.f90, for example (*do not* name them after the PR number of the bug you fix, please).

Then, at the beginning of each one, write a very short comment to describe what it tests (one line can be enough), as well as the associated PR number if it's a bugfix. Something like: 

! Testcase for the -ffree-line-length-none option
! See PR fortran/42

Then, you need to put dejagnu directives. A directive is a { } enclosed text inside a comment, either at the beginning of the program, or on a given line of the program.

dg-do

You should first put a directive indicating dejagnu what to do for the test; possibilities are: 

! { dg-do compile }
! { dg-do link }
! { dg-do run }
(Do not forget the spaces within the curly brackets!)

Given that, dejagnu will try to, respectively, compile, compile + link or compile + link + execute the program with the -pedantic option (and various optimisation options for execution tests) and report a FAILure if the compilation or the execution fails.

dg-options

If you don't want dejagnu to compile the program with -pedantic use the { dg-options "-fbackslash" } directive to replace it by -fbackslash for example. You can specify more than one option: 

! { dg-options "-fbackslash -std=legacy" }
or zero, just to prevent the -pedantic to step in:
! { dg-options "" }

dg-warning, dg-error

By default, dejagnu will mark a test as FAILed if a warning is emitted during compilation. If you expect a warning to be raised (with option -pedantic remember), write on the line of code that triggers it: 

print *, modulo(4_4,3_8) ! { dg-warning "same type and kind" }

where the argument to dg-warning is a regular expression for a part of the warning message. Similarly, there is a dg-error directive.

Should a line produce two errors, the regular expression can include an "|" (ie. a regular expression OR) between the possible message fragments. (See, for example, gfortran.dg/external_implicit_none.f90 or implicit_actual.f90.)

If only one of the errors is important, a better way to deal with the multiple errors is to check for the one you want with dg-error and discard the extras with dg-prune-output: 

! { dg-prune-output "redundant error" }

This way the test will fail if the important error message goes away, but the extras can come and go without affecting test results.

dg-final

For Fortran test cases which contain modules (let's say they are named 'm1', 'm2' and 'm3'), the module files should be cleaned up via: 

! { dg-final { cleanup-modules "m1 m2 m3" } }

Sometimes it is necessary to scan the tree dump for the occurrence of certain strings. Example: To verify that the output of -fdump-tree-original contains the string 'bla_bla' exactly five times (and clean up the dump afterwards), do: 

! { dg-do compile }
! { dg-options "-fdump-tree-original" }
...
! { dg-final { scan-tree-dump-times "bla_bla" 5 "original" } }
! { dg-final { cleanup-tree-dump "original" } }

More tricks

The best way to learn all this is to grep and look at the existing gfortran.dg testcases, trying things and read the log of the testsuite (${builddir]/gcc/testsuite/gfortran.log). One particularly interesting trick: to run only certain testcases, you can use: 

make check-gfortran RUNTESTFLAGS=dg.exp=my_test_*.f*

(for more details, see the doc on http://gcc.gnu.org/install/test.html)

Other advanced features not discussed here:

Happy hacking!

For more information see also: HowToPrepareATestcase

None: TestCaseWriting (last edited 2012-06-20 17:04:45 by JasonMerrill)