Pytest usage in DejaGNU?

[Martin Liška]

Hello.

GCOV tests suffer from tests that would cover the intermediate format.
It's a JSON format and and I'm attaching an example of its output.

I would really like to use Python to make more complex tests:

$ cat test_json.py
import pytest
import json

def test_gcov_output():
     data = json.load(open('gcov.json'))
     assert len(data['files']) == 1
     f0 = data['files'][0]
     assert f0['file'] == 'gcov-lambda.C'
     assert len(f0['functions']) == 3

     fns = {}
     for fn in f0['functions']:
         fns[fn['name']] = fn
     lines = f0['lines']

     for line in lines:
         lineno = line['line_number']
         linefn = line['function_name']
         assert linefn in fns
         fn = fns[linefn]
         assert fn['start_line'] <= lineno and lineno <= fn['end_line']

I see it pretty complicated to do the same in DejaGNU. Mainly due the missing
JSON parser.

Would it be possible to make optional Python tests in our testsuite?
I can imagine a simple pytest wrapper that will do something like:

+proc pytest-execute { dgargs } {
+    verbose "dg-pytest-execute: ${dgargs}" 2
+    set script [lindex $dgargs 0]
+    verbose "  script: ${script}" 2
+
+    spawn -noecho pytest -rA -s --tb=no $script
+
+    expect {
+      -re "FAILED .*" {
+       fail "pytest $expect_out(0,string)"
+      }
+      -re "PASSED .*" {
+       pass "pytest $expect_out(0,string)"
+      }
+    }
+}

as Pytest can provide a reasonable close output:

=========================================================================================================================== short test summary info ============================================================================================================================
PASSED test_json.py::test_gcov_output
PASSED test_json.py::test_gcov_output
PASSED test_json.py::test_gcov_output

Thoughts?
Martin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gcov.json
Type: application/json
Size: 4817 bytes
Desc: not available
URL: <https://gcc.gnu.org/pipermail/gcc/attachments/20201214/efa41bd7/attachment.json>