Re: [PATCH/RFC]: unittesting v2: as a plugin (was Re: [PATCH 00/17] RFC: Addding a unit testing framework to gcc)

[Jeff Law <law at redhat dot com>]

On 06/17/2015 01:48 PM, David Malcolm wrote:
> On Thu, 2015-06-11 at 00:18 +0200, Jakub Jelinek wrote:
>
> I wrote a custom formatter for the output (class deja_gnu_printer within
> unittests-plugin.c) which generates lines like this on stderr:
>
> PASS: ggc_test.tree_marking
> PASS: ggc_test.custom_struct
> PASS: ggc_test.finalization
> PASS: ggc_test.inheritance
> PASS: ggc_test.chain_next
>
> These get detected on stderr by some new logic inside
> testsuite/lib/prune.exp, which prefixes them with
> [testname-for-summary], and emitting them at the Tcl level, so they end
> up in the .log and .sum files as lines like this:
>
> PASS: c-c++-common/torture/run-unittests-plugin.c   -O0   ggc_test.tree_marking
> PASS: c-c++-common/torture/run-unittests-plugin.c   -O0   ggc_test.custom_struct
> PASS: c-c++-common/torture/run-unittests-plugin.c   -O0   ggc_test.finalization
> PASS: c-c++-common/torture/run-unittests-plugin.c   -O0   ggc_test.inheritance
> PASS: c-c++-common/torture/run-unittests-plugin.c   -O0   ggc_test.chain_next
>
> (if doing it for all testcase output is an issue, perhaps this prune.exp
> logic could be made conditional, so that we only do it for testcases
> using the plugin, or that contain a dg command)
Yea, this is roughly what I was expecting -- you don't have access to 
the .log/.sum files from within the test, so you have to scan the output 
from the test and emit the pass/fail/xfail/whatever messages.

Presumably there's a mechanism for mark expected failures in the gtest 
framework that we can turn into XFAIL/XPASS as needed.  What about the 
UNSUPPORTED result -- perhaps useful if a test depends on some attribute 
of the target.


> This works, and means there's no extra external dependency, and I was
> able to bootstrap with this.  I ran into an issue with "make check": the
> plugin is linked against the previous stage's libstdc++, but "make
> check" doesn't seem to set up LD_LIBRARY_PATH to point at a fresh
> libstdc++ and uses the system copy, leading to
> cc1: error: cannot load plugin ../../unittests_plugin.so
> /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required
> by ../../unittests_plugin.so)
>
> on this Fedora 20 box.  Manually setting LD_LIBRARY_PATH to point at the
> built libstdc++ fixes it.
You might look at the libstdc++ testsuite.  It has to arrange to get the 
just built library used for testing.

Jeff