beginner's guide to gfortran testsuite (was: Re: [patch] pr21302: Max line length in free form mode)

[FX Coudert]

[ Post-scriptum: this mail turned into a general beginner's guide to 
writing gfortran testcases, because I remember having a hard time with 
it when I first had to work on this. If someone wants to augment it, 
correct the mistakes I made, and then post it on the Wiki, it might be 
worth the trouble. ]

> The testcases are still valid:
> http://gcc.gnu.org/ml/fortran/2005-11/msg00551.html

For the testcases, they need to be put into suitable dejagnu framework 
form. From the examples you gave, we can make two independent testcases.

* Give them a non-equivocal name, ending with an underscore and a 
number: line_length_1.f and line_length_1.f90, for example (*do not* 
name them after the PR number of the bug you fix, please).


* Then, at the beginning of each one, write a very short description of 
what it tests (one line can be enough), as well as the associated PR 
number if it's a bugfix. Something like:

! Testcase for the -ffree-line-length-none option
! See PR fortran/42


* Then, you need to put dejagnu directives. A directive is a { } 
enclosed text inside a comment, either at the beginning of the program, 
or on a given line of the program. You should first put a directive 
indicating dejagnu what to do for testing; possibilities are:

! { dg-do compile }
! { dg-do link }
! { dg-do execute }

(Mind the spaces around the curly brackets!) Given that, dejagnu will 
try to compile (resp. compile and link, compile and link and execute) 
the program with a -pedantic option (and various optimisation options 
for execution tests) and report the test a FAILing if the compilation or 
the execution fails.

* If you don't want dejagnu to compile the program with -pedantic, use 
the { dg-options "-fbackslash" } directive to replace it by -fbackslash, 
for example. You can specify more than one option:

! { dg-options "-fbackslash -std=legacy" }

or zero, just to prevent the -pedantic to step in:

! { dg-options "" }


* By default, dejagnu will mark a test as FAILed if a warning is emitted 
during compilation. If you expect a warning to be raised (with option 
-pedantic, remember), write on the line of code that triggers it:

   print *, modulo(4_4,3_8) ! { dg-warning "same type and kind" }

where the argument to dg-warning is a part of (actually, a regular 
expression for a part of) the warning message. Similarly, there is a 
dg-error directive.


* The best way to learn all this is to grep and look at the existing 
gfortran.dg testcases, trying things and read the log of the testsuite 
(${builddir]/gcc/testsuite/gfortran.log). One particularly interesting 
trick: to run only certain testcases, you can use:

$ make check-gfortran RUNTESTFLAGS="dg.exp=gfortran.dg/my_test_*.f*"

(for more details, see the doc on http://gcc.gnu.org/install/test.html)


* Other advanced features not discussed here: dg-additional, which 
enables us to create mixed C-Fortran testcases; dg-output, which tests 
for the output of the executed program; dg-require-* to run a testcase 
on targets fulfilling certain conditions; and { xfail *-*-* } 
directives, for test that are expected to fail on certain platforms.


Happy hacking,
FX