[PATCH 00/12] RFC: Replay of serialized diagnostics
David Malcolm
dmalcolm@redhat.com
Wed Jun 22 22:34:35 GMT 2022
We currently have a couple of formats into which our diagnostics can
be serialized: (a) gcc's own json format and (b) SARIF, via:
-fdiagnostics-format=json-stderr (and -fdiagnostics-format=json)
-fdiagnostics-format=json-file
-fdiagnostics-format=sarif-stderr
-fdiagnostics-format=sarif-file
This experimental patch kit implements the ability for GCC to
replay these serialized diagnostics - and thus to use GCC's diagnostic
printing machinery to print the output of any tool that can write to
these formats.
It implements the replayers as gcc frontends, handling .json and .sarif
files, so that in theory you can write:
gcc foo.sarif
and have gcc replay things. Unfortunately that doesn't quite work: you
have to tell it -S, and if the file isn't found, it thinks it's a linker
script.
For example, here's the output of replaying some of the files from the
SARIF tutorial [https://github.com/microsoft/sarif-tutorials]:
e.g. the results of a javascript linter:
$ ./xgcc -B. -S ../../../sarif-tutorials/samples/1-Introduction/simple-example.sarif
file:///C:/dev/sarif/sarif-tutorials/samples/Introduction/simple-example.js:1:5: error: 'x' is assigned a value but never used. [no-unused-vars]
(the no-unused-vars is actually a hyperlink to https://eslint.org/docs/rules/no-unused-vars in my terminal)
and a Python analyzer:
$ ./xgcc -B. -S -xsarif ../../../sarif-tutorials/samples/3-Beyond-basics/bad-eval-with-code-flow.sarif
3-Beyond-basics/bad-eval-with-code-flow.py:8:1: warning: Use of tainted variable 'raw_input' in the insecure function 'eval'. [PY2335]
events 1-2
|
|
+--> event 3
|
|
As you can see, it has some issues finding source code when replaying
things (there's an example in the final patch in the kit that manages
this).
Thoughts?
Would it be better to have an explicit tool for this
(e.g. "gcc-replay-sarif"), rather than have it as a frontend?
Doesn't yet bootstrap, and has FIXMEs and TODOs but posting now in the
hope of getting feedback.
David Malcolm (12):
diagnostics: add ability to associate diagnostics with rules from
coding standards
diagnostics: associate rules with plugins in SARIF output
Add more emit_diagnostic overloads
json: add json parsing support
Placeholder libcpp fixups
prune.exp: move multiline-handling to before other pruning
Add deferred-locations.h/cc
Add json-reader.h/cc
Add json frontend
Add sarif frontend
Fixups to diagnostic-format-sarif.cc
Work-in-progress of path remapping
gcc/Makefile.in | 2 +-
gcc/common.opt | 4 +
gcc/deferred-locations.cc | 231 ++
gcc/deferred-locations.h | 52 +
gcc/diagnostic-client-data-hooks.h | 18 -
gcc/diagnostic-client-plugin.h | 43 +
gcc/diagnostic-core.h | 7 +
gcc/diagnostic-format-json.cc | 1 +
gcc/diagnostic-format-sarif.cc | 303 ++-
gcc/diagnostic-metadata.h | 65 +-
gcc/diagnostic.cc | 73 +
gcc/diagnostic.h | 3 +
gcc/doc/invoke.texi | 10 +
gcc/doc/plugins.texi | 17 +-
gcc/input.cc | 107 +-
gcc/input.h | 18 +-
gcc/json-parsing.cc | 2391 +++++++++++++++++
gcc/json-parsing.h | 94 +
gcc/json-reader.cc | 122 +
gcc/json-reader.h | 107 +
gcc/json.cc | 2 +-
gcc/json.h | 59 +-
gcc/json/Make-lang.in | 131 +
gcc/json/config-lang.in | 34 +
gcc/json/json-frontend.cc | 176 ++
gcc/json/json-replay.cc | 614 +++++
gcc/json/json-replay.h | 26 +
gcc/json/lang-specs.h | 26 +
gcc/json/lang.opt | 31 +
gcc/opts.cc | 4 +
gcc/plugin.cc | 36 +-
gcc/plugin.h | 12 +-
gcc/sarif/Make-lang.in | 132 +
gcc/sarif/config-lang.in | 34 +
gcc/sarif/lang-specs.h | 26 +
gcc/sarif/lang.opt | 31 +
gcc/sarif/sarif-frontend.cc | 191 ++
gcc/sarif/sarif-replay.cc | 1528 +++++++++++
gcc/sarif/sarif-replay.h | 26 +
gcc/selftest-run-tests.cc | 1 +
gcc/selftest.h | 1 +
.../plugin/diagnostic-test-metadata-sarif.c | 39 +
.../gcc.dg/plugin/diagnostic-test-metadata.c | 2 +-
.../plugin/diagnostic_plugin_test_metadata.c | 13 +-
gcc/testsuite/gcc.dg/plugin/plugin.exp | 4 +-
.../invalid-json-array-missing-comma.json | 6 +
...nvalid-json-array-with-trailing-comma.json | 6 +
.../json/invalid-json-bad-token.json | 6 +
.../invalid-json-object-missing-comma.json | 7 +
...valid-json-object-with-trailing-comma.json | 6 +
.../invalid-jsondump-diag-not-an-object.json | 6 +
.../invalid-jsondump-kind-not-a-string.json | 20 +
.../json/invalid-jsondump-not-an-array.json | 6 +
gcc/testsuite/json/json.exp | 50 +
gcc/testsuite/json/signal-1.c.json | 131 +
gcc/testsuite/lib/json-dg.exp | 233 ++
gcc/testsuite/lib/json.exp | 36 +
gcc/testsuite/lib/prune.exp | 14 +-
gcc/testsuite/lib/sarif-dg.exp | 233 ++
gcc/testsuite/lib/sarif.exp | 36 +
.../sarif/bad-eval-with-code-flow.py | 10 +
gcc/testsuite/sarif/escaped-braces.sarif | 19 +
.../invalid-json-array-missing-comma.sarif | 6 +
...valid-json-array-with-trailing-comma.sarif | 6 +
.../sarif/invalid-json-bad-token.sarif | 6 +
.../invalid-json-object-missing-comma.sarif | 7 +
...alid-json-object-with-trailing-comma.sarif | 6 +
.../sarif/invalid-sarif-bad-runs.sarif | 7 +
...f-missing-arguments-for-placeholders.sarif | 14 +
.../sarif/invalid-sarif-no-runs.sarif | 6 +
.../sarif/invalid-sarif-no-version.sarif | 6 +
.../invalid-sarif-non-object-in-runs.sarif | 7 +
.../sarif/invalid-sarif-not-an-object.sarif | 6 +
...ot-enough-arguments-for-placeholders.sarif | 14 +
.../invalid-sarif-version-not-a-string.sarif | 6 +
.../sarif/malformed-placeholder.sarif | 15 +
gcc/testsuite/sarif/null-runs.sarif | 2 +
.../sarif/roundtrip-signal-1.c.sarif | 398 +++
gcc/testsuite/sarif/sarif.exp | 50 +
gcc/testsuite/sarif/signal-1.c.sarif | 362 +++
gcc/testsuite/sarif/spec-example-1.sarif | 15 +
gcc/testsuite/sarif/spec-example-2.sarif | 74 +
gcc/testsuite/sarif/spec-example-3.sarif | 67 +
gcc/testsuite/sarif/spec-example-4.sarif | 758 ++++++
.../sarif/tutorial-example-foo.sarif | 130 +
gcc/toplev.cc | 2 +
gcc/tree-diagnostic-client-data-hooks.cc | 36 +-
libcpp/include/line-map.h | 7 +-
libcpp/line-map.cc | 3 +-
89 files changed, 9500 insertions(+), 188 deletions(-)
create mode 100644 gcc/deferred-locations.cc
create mode 100644 gcc/deferred-locations.h
create mode 100644 gcc/diagnostic-client-plugin.h
create mode 100644 gcc/json-parsing.cc
create mode 100644 gcc/json-parsing.h
create mode 100644 gcc/json-reader.cc
create mode 100644 gcc/json-reader.h
create mode 100644 gcc/json/Make-lang.in
create mode 100644 gcc/json/config-lang.in
create mode 100644 gcc/json/json-frontend.cc
create mode 100644 gcc/json/json-replay.cc
create mode 100644 gcc/json/json-replay.h
create mode 100644 gcc/json/lang-specs.h
create mode 100644 gcc/json/lang.opt
create mode 100644 gcc/sarif/Make-lang.in
create mode 100644 gcc/sarif/config-lang.in
create mode 100644 gcc/sarif/lang-specs.h
create mode 100644 gcc/sarif/lang.opt
create mode 100644 gcc/sarif/sarif-frontend.cc
create mode 100644 gcc/sarif/sarif-replay.cc
create mode 100644 gcc/sarif/sarif-replay.h
create mode 100644 gcc/testsuite/gcc.dg/plugin/diagnostic-test-metadata-sarif.c
create mode 100644 gcc/testsuite/json/invalid-json-array-missing-comma.json
create mode 100644 gcc/testsuite/json/invalid-json-array-with-trailing-comma.json
create mode 100644 gcc/testsuite/json/invalid-json-bad-token.json
create mode 100644 gcc/testsuite/json/invalid-json-object-missing-comma.json
create mode 100644 gcc/testsuite/json/invalid-json-object-with-trailing-comma.json
create mode 100644 gcc/testsuite/json/invalid-jsondump-diag-not-an-object.json
create mode 100644 gcc/testsuite/json/invalid-jsondump-kind-not-a-string.json
create mode 100644 gcc/testsuite/json/invalid-jsondump-not-an-array.json
create mode 100644 gcc/testsuite/json/json.exp
create mode 100644 gcc/testsuite/json/signal-1.c.json
create mode 100644 gcc/testsuite/lib/json-dg.exp
create mode 100644 gcc/testsuite/lib/json.exp
create mode 100644 gcc/testsuite/lib/sarif-dg.exp
create mode 100644 gcc/testsuite/lib/sarif.exp
create mode 100644 gcc/testsuite/sarif/bad-eval-with-code-flow.py
create mode 100644 gcc/testsuite/sarif/escaped-braces.sarif
create mode 100644 gcc/testsuite/sarif/invalid-json-array-missing-comma.sarif
create mode 100644 gcc/testsuite/sarif/invalid-json-array-with-trailing-comma.sarif
create mode 100644 gcc/testsuite/sarif/invalid-json-bad-token.sarif
create mode 100644 gcc/testsuite/sarif/invalid-json-object-missing-comma.sarif
create mode 100644 gcc/testsuite/sarif/invalid-json-object-with-trailing-comma.sarif
create mode 100644 gcc/testsuite/sarif/invalid-sarif-bad-runs.sarif
create mode 100644 gcc/testsuite/sarif/invalid-sarif-missing-arguments-for-placeholders.sarif
create mode 100644 gcc/testsuite/sarif/invalid-sarif-no-runs.sarif
create mode 100644 gcc/testsuite/sarif/invalid-sarif-no-version.sarif
create mode 100644 gcc/testsuite/sarif/invalid-sarif-non-object-in-runs.sarif
create mode 100644 gcc/testsuite/sarif/invalid-sarif-not-an-object.sarif
create mode 100644 gcc/testsuite/sarif/invalid-sarif-not-enough-arguments-for-placeholders.sarif
create mode 100644 gcc/testsuite/sarif/invalid-sarif-version-not-a-string.sarif
create mode 100644 gcc/testsuite/sarif/malformed-placeholder.sarif
create mode 100644 gcc/testsuite/sarif/null-runs.sarif
create mode 100644 gcc/testsuite/sarif/roundtrip-signal-1.c.sarif
create mode 100644 gcc/testsuite/sarif/sarif.exp
create mode 100644 gcc/testsuite/sarif/signal-1.c.sarif
create mode 100644 gcc/testsuite/sarif/spec-example-1.sarif
create mode 100644 gcc/testsuite/sarif/spec-example-2.sarif
create mode 100644 gcc/testsuite/sarif/spec-example-3.sarif
create mode 100644 gcc/testsuite/sarif/spec-example-4.sarif
create mode 100644 gcc/testsuite/sarif/tutorial-example-foo.sarif
--
2.26.3
More information about the Gcc-patches
mailing list