Understand GCC test process

[Martin Sebor]

On 10/07/2015 07:57 AM, Sabrina Souto wrote:
> I was seeing these files but I could not put the puzzle pieces
> together in my mind, and after you explained, all pieces make sense
> now. Thanks for the explanation, Jonathan.
> The testing process is clear now, but I still not understanding what
> can explain that amount of common functions (over 60%) across all
> traces of the tests (that I analyzed) from gcc.dg test suite, could
> you help me with this? I have some hypotheses:
> (1) All tests run the compiler and there is a large "core" code that
> is executed regardless of the compiler option?
> (2) Since all tests run the same driver - dg.exp, I could say that
> this great amount of common functions is due to what is specified in
> the driver, and the remaining few different functions are related with
> the particularity of each test?
> (3) None of those. Do you have another explanation?

I'm not sure what you mean by "function" (compiler invocations?)
or "trace" (verbose DejaGnu output?) but perhaps what you are
referring to by "common functions across all traces" is the test
driver invoking the compiler and running tests to figure out what
features are supported on the target.

For example, running just the simple array-1.c test like so:

   $ make RUNTESTFLAGS='dg.exp=array-1.c -v -v' check-gcc

will show something like:

   ...
   check_compile tool: gcc for lto
   doing compile
   Invoking the compiler as ...
   ...
   check_compile tool: gcc for linker_plugin
   doing compile
   Invoking the compiler as ...
   ...
   Testing gcc.dg/array-1.c
   doing compile
   Invoking the compiler as...

More involved tests (those with more complex DejaGnu directives)
might cause the driver to run other tests. The results of these
should be cached so subsequent tests that depend on the results
don't cause the same tests to be run again.

If by the "function" you mean a DejaGnu function, you need to look
at the .exp files to see what's invoked when: your local runtest.exp
which is invoked first, and then GCC's dg.exp (and anything in
between). You can see the pathnames of all the .exp files in the
DejaGnu verbose output (-v -v).

The best way to understand how it all works is to read the .exp
files and experiment with some increasingly more involved use cases,
study their verbose output and the .log files in the corresponding
testsuire subdirectory.

Martin