Re: [PATCH 0/2] Python testcases to check DWARF output

[Jeff Law <law at redhat dot com>]

On 07/26/2017 10:00 AM, Pierre-Marie de Rodat wrote:
> Hello,
> 
> At the last GNU Cauldron, Richard Biener and I talked about DWARF output
> testing. Except for guality tests, which are disabled on several
> targets, the only way tests check the DWARF is scanning the annotated
> assembly (-dA), making it hard to write reliable tests.
> 
> For instance, checking the number of times DW_AT_location is present in
> order to check that some specific variable is assigned one is fuzzy.
> Depending on the target and on the evolution of the compiler, the number
> of output variables, or which one is assigned a location can vary
> legitimately but still make the test fail.
> 
> On my side, I already had written an out-of-tree testsuite for the DWARF
> features I added for Ada. This testsuite uses a DWARF parser in order to
> perform checks on a tree:
> <https://github.com/pmderodat/dwarf-ada-testsuite/>. I had to update it
> a couple of times, for instance when a change created a
> DW_TAG_const_type DIE or removed one somewhere in a type tree, but
> that’s very rare. I would say that I’m satisfied with the checks I could
> express, but I don’t remember I ever caught a regression with them, so I
> have no representative experience to share in this area. Maybe DWARF
> back-end developpers do a too good job. ;-)
> 
> Anyway, Richard and I discussed about doing something similar in-tree,
> and here is a candidate set of patches to achieve that:
> 
>   * The first patch installs DejaGNU scripts to run a Python interpreter
>     in testcases.
> 
>   * The second one installs other DejaGNU scripts to detect DWARF
>     dumping tools, plus a small Python library to parse and pattern
>     match DIEs and their attributes. It also adds several C and Ada
>     tests as examples; these are inspired by existing homonym tests
>     based on assembly scanning.
> 
> For now, this supports only platforms where objdump is available for the
> current target, but extending it to other tools, such as otool on Darwin
> should be doable.
> 
> I would appreciate feedback about the idea and the implementation I
> propose. This is the first time I do more in the testsuite than just
> adding new tests, so thank you in advance for you patience in reviewing
> these. :-)
> 
> I tested these patches on x86_64-linux.
I hate to throw in a wrench at this point, but has anyone looked at
dwgrep from Petr Machata?  He's not doing much with it anymore, but it
might provide enough of a dwarf scanning framework to be useful for
testing purposes.

jeff