Beginner's Guide to Writing Testcases

Give your testcases an unequivocal name, ending with an underscore and a number: line_length_1.f and line_length_1.f90, for example (*do not* name them after the PR number of the bug you fix, please).

Then, at the beginning of each one, write a very short comment to describe what it tests (one line can be enough), as well as the associated PR number if it's a bugfix. Something like:

! Testcase for the -ffree-line-length-none option
! See PR fortran/42

Then, you need to put dejagnu directives. A directive is a { } enclosed text inside a comment, either at the beginning of the program, or on a given line of the program.

dg-do

You should first put a directive indicating dejagnu what to do for the test; possibilities are:

! { dg-do compile }
! { dg-do link }
! { dg-do run }
(Do not forget the spaces within the curly brackets!)

Given that, dejagnu will try to, respectively, compile, compile + link or compile + link + execute the program with the -pedantic option (and various optimisation options for execution tests) and report a FAILure if the compilation or the execution fails.

dg-options

If you don't want dejagnu to compile the program with -pedantic use the { dg-options "-fbackslash" } directive to replace it by -fbackslash for example. You can specify more than one option:

! { dg-options "-fbackslash -std=legacy" }
or zero, just to prevent the -pedantic to step in:
! { dg-options "" }

dg-warning, dg-error

By default, dejagnu will mark a test as FAILed if a warning is emitted during compilation. If you expect a warning to be raised (with option -pedantic remember), write on the line of code that triggers it:

print *, modulo(4_4,3_8) ! { dg-warning "same type and kind" }

where the argument to dg-warning is a regular expression for a part of the warning message. Similarly, there is a dg-error directive.

Should a line produce two errors, the regular expression can include an "|" (ie. a regular expression OR) between the possible message fragments. (See, for example, gfortran.dg/external_implicit_none.f90 or implicit_actual.f90.)

If only one of the errors is important, a better way to deal with the multiple errors is to check for the one you want with dg-error and discard the extras with dg-prune-output:

! { dg-prune-output "redundant error" }

This way the test will fail if the important error message goes away, but the extras can come and go without affecting test results.

dg-final

For Fortran test cases which contain modules (let's say they are named 'm1', 'm2' and 'm3'), the module files can be cleaned up via:

! { dg-final { cleanup-modules "m1 m2 m3" } }

However, this is not necessary any more, because cleaning up module files is done automatically now.

Sometimes it is necessary to scan the tree dump for the occurrence of certain strings. Example: To verify that the output of -fdump-tree-original contains the string 'bla_bla' exactly five times (and clean up the dump afterwards), do:

! { dg-do compile }
! { dg-options "-fdump-tree-original" }
...
! { dg-final { scan-tree-dump-times "bla_bla" 5 "original" } }
! { dg-final { cleanup-tree-dump "original" } }

More tricks

The best way to learn all this is to grep and look at the existing gfortran.dg testcases, trying things and read the log of the testsuite (${builddir}/gcc/testsuite/gfortran.log). One particularly useful trick is to run only certain testcases by making the check-fortran target (in ${builddir}, or equivalently, the check-gfortran target in the ${builddir}/gcc subdirectory). For example, to run all Fortran tests whose name matches the my_test_*.f* shell pattern, invoke make as follows:

make -C ${builddir} check-fortran RUNTESTFLAGS="dg.exp=my_test_*.f*"

(for more details, see the doc on http://gcc.gnu.org/install/test.html)

Other advanced features not discussed here:

Happy hacking!

For more information see also:

None: TestCaseWriting (last edited 2016-11-09 19:15:52 by JanusWeil)