Beginner's Guide to Writing Testcases

Give your testcases an unequivocal name, ending with an underscore and a number: line_length_1.f and line_length_1.f90, for example (*do not* name them after the PR number of the bug you fix, please).

Then, at the beginning of each one, write a very short comment to describe what it tests (one line can be enough), as well as the associated PR number if it's a bugfix. Something like: 

! Testcase for the -ffree-line-length-none option
! See PR fortran/42

Then, you need to put dejagnu directives. A directive is a { } enclosed text inside a comment, either at the beginning of the program, or on a given line of the program.

dg-do

You should first put a directive indicating dejagnu what to do for the test; possibilities are: 

! { dg-do compile }
! { dg-do link }
! { dg-do run }
(Do not forget the spaces within the curly brackets!)

Given that, dejagnu will try to, respectively, compile, compile + link or compile + link + execute the program with the -pedantic option (and various optimisation options for execution tests) and report a FAILure if the compilation or the execution fails.

dg-options

If you don't want dejagnu to compile the program with -pedantic use the { dg-options "-fbackslash" } directive to replace it by -fbackslash for example. You can specify more than one option: 

! { dg-options "-fbackslash -std=legacy" }
or zero, just to prevent the -pedantic to step in:
! { dg-options "" }

dg-warning, dg-error

By default, dejagnu will mark a test as FAILed if a warning is emitted during compilation. If you expect a warning to be raised (with option -pedantic remember), write on the line of code that triggers it: 

print *, modulo(4_4,3_8) ! { dg-warning "same type and kind" }

where the argument to dg-warning is a regular expression for a part of the warning message. Similarly, there is a dg-error directive.

Should a line produce two errors, the regular expression can include an "|" (ie. a regular expression OR) between the possible message fragments. (See, for example, gfortran.dg/external_implicit_none.f90 or implicit_actual.f90.)

If only one of the errors is important, a better way to deal with the multiple errors is to check for the one you want with dg-error and discard the extras with dg-prune-output: 

! { dg-prune-output "redundant error" }

This way the test will fail if the important error message goes away, but the extras can come and go without affecting test results.

dg-final

Sometimes it is necessary to scan the tree dump for the occurrence of certain strings. Example: To verify that the output of -fdump-tree-original contains the string 'bla_bla' exactly five times (and clean up the dump afterwards), do: 

! { dg-do compile }
! { dg-options "-fdump-tree-original" }
...
! { dg-final { scan-tree-dump-times "bla_bla" 5 "original" } }

cleanup

For Fortran test cases which contain modules (let's say they are named 'm1', 'm2' and 'm3'), the module files can be cleaned up via: 

! { dg-final { cleanup-modules "m1 m2 m3" } }

However, this is not necessary any more, because cleaning up module files is done automatically now. Also dump files are cleaned up automatically by now, so that lines like these are deprecated: 

! { dg-final { cleanup-tree-dump "original" } }

Running Fortran tests

In order to run all Fortran test cases in the GCC repository, one needs to execute the following command in the build directory: 

make -k check-fortran

On a multi-CPU machine, it is advisable to run the tests in parallel, for example by using 

make -j8 -k check-fortran

If you are on a x86_64 machine and want to run both 32- and 64-bit tests, you can do 

make -k -j8 check-fortran RUNTESTFLAGS="--target_board=unix'{-m32,-m64}'"

To run only a certain subset, e.g. only tests whose name matches the pattern my_test_*, one can invoke make as follows: 

make -k check-fortran RUNTESTFLAGS="dg.exp=my_test_*"

Note that check-fortran also contains the Fortran tests from libgomp, which can be run by themselves like this: 

make -k check-target-libgomp RUNTESTFLAGS=fortran.exp
make -k check-target-libgomp RUNTESTFLAGS="fortran.exp=my_test_*"

(for more details, see the doc on http://gcc.gnu.org/install/test.html)

More tricks

The best way to learn all this is to grep and look at the existing gfortran.dg testcases, trying things and read the log of the testsuite (${builddir}/gcc/testsuite/gfortran.log).

Other advanced features not discussed here:

Happy hacking!

For more information see also:

None: TestCaseWriting (last edited 2019-04-28 12:48:59 by ThomasKoenig)