Hundreds of gcc.dg/guality failures on both 14 and 13.1 branches

[Jakub Jelinek]

On Sat, Jul 15, 2023 at 10:58:40PM +0200, Martin Jambor wrote:
> Hi,
> 
> On Sat, Jul 15 2023, FX Coudert via Gcc wrote:
> > Hi,
> >
> > I am finding it very hard to reliably compare test results and regressions with the very large number of gcc.dg/guality test failures that are apparently the new norm on x86_64-linux: more than one hundred on 13.1, and several hundreds on 14. Is there any on-going discussion about this?
> >
> > I mean, from an almost-external point of view, these tests should probably be xfail'ed and a PR opened against them to reenable them.
> >
> 
> As far as I understand it, the main problem is that it is not really
> possible to XFAIL a test for one combination of options (say "-O2
> -flto") and not others.

It is far worse than that.  We have the target vs. different ISA options
vs. different -O? options vs. different versions of gdb matrix and maintaining
what cases from those are supposed to pass and what are expected to fail
is really hard.
Which is why people should just compare testsuite results from earlier run
on the same configuration to watch for regressions, especially in the
guality testsuite.

	Jakub