This is the mail archive of the
mailing list for the GCC project.
Re: [PATCH] Don't run guality.exp tests with LTO_TORTURE_OPTIONS.
- From: Richard Biener <rguenther at suse dot de>
- To: Mark Wielaard <mjw at redhat dot com>,Jakub Jelinek <jakub at redhat dot com>
- Cc: Richard Biener <richard dot guenther at gmail dot com>,Jason Merrill <jason at redhat dot com>,gcc-patches at gcc dot gnu dot org,Cary Coutant <ccoutant at google dot com>,Alexandre Oliva <aoliva at redhat dot com>,Diego Novillo <dnovillo at google dot com>
- Date: Fri, 04 Jul 2014 10:12:26 +0200
- Subject: Re: [PATCH] Don't run guality.exp tests with LTO_TORTURE_OPTIONS.
- Authentication-results: sourceware.org; auth=none
- References: <1404295545-17107-1-git-send-email-mjw at redhat dot com> <53B49076 dot 8050005 at redhat dot com> <20140703053713 dot GZ31640 at tucnak dot redhat dot com> <b305ae42-a9e9-42ce-bd9b-d3e444bc3441 at email dot android dot com> <20140703075536 dot GB31640 at tucnak dot redhat dot com> <30d70fa5-e04b-4526-8e32-fa01298baa24 at email dot android dot com> <20140703183814 dot GJ31640 at tucnak dot redhat dot com> <0a86fabc-9388-4b11-89c6-fc09f6b4d9af at email dot android dot com> <1404417875 dot 3766 dot 76 dot camel at bordewijk dot wildebeest dot org> <20140703201400 dot GN31640 at tucnak dot redhat dot com> <1404420472 dot 3766 dot 84 dot camel at bordewijk dot wildebeest dot org>
On July 3, 2014 10:47:52 PM CEST, Mark Wielaard <firstname.lastname@example.org> wrote:
>On Thu, 2014-07-03 at 22:14 +0200, Jakub Jelinek wrote:
>> On Thu, Jul 03, 2014 at 10:04:35PM +0200, Mark Wielaard wrote:
>> > On Thu, 2014-07-03 at 21:52 +0200, Richard Biener wrote:
>> > > On July 3, 2014 8:38:14 PM CEST, Jakub Jelinek <email@example.com>
>> > > >On Thu, Jul 03, 2014 at 08:37:07PM +0200, Richard Biener wrote:
>> > > >> Well, simply removing the regression testing for LTO is a
>> > > >maintainance nightmare as well.
>> > > >>
>> > > >> The guality testsuite is very noisy anyway with all the xfail
>> > > >xpass.
>> > > >
>> > > >Let's keep it as is then?
>> > >
>> > > That works for me.
>> > I don't find that very satisfactory. I want to add more guality
>> > but the fact that they are unreliable and by default introduce even
>> > FAILs when lto is enabled makes that not very attractive. I do like
>> > Jakub's suggestion to disable the guality tests be run with lto by
>> > default, but provide an environment variable to enable them for
>> > that want to try them anyway. Shall I implement that?
>> They aren't that unrealiable (at least, if people committing patches
>> ignore regressions in there). Just one should diff
>> output from earlier builds to the latest, that way it is clear what
>> regression and what is not.
>The are much more unreliable than any other test. With guality.exp
>disabled one can just eyeball the results and investigate new FAILS.
>There are only a handful. When you include guality.exp you can easily
>get the impression the gcc testsuite is really bad (and it isn't!) And
>the problem is that it makes adding new tests a pain. See my new tests,
>they introduce new FAILs because LTO is enabled by default for
>guality.exp at the moment. It just results in a slow increase of FAILs
>that people have to ignore. And I am afraid that will just result in
>people missing real regressions.
>I don't mind if there is active work to fix LTO DWARF debuginfo
>generation issues and the guality.exp LTO failures will soon disappear,
>but if there is no active work on reducing the amount of failures and
>introducing new guality.exp testcases will keep adding more FAILs I
>think we are much better off disabling them for now.
Can't you simply add proper xfails for lto when you add new tests that fail with lto?
(Please quickly check why - usually the tests are just optimized in an unexpected way - compare with -fwhole-program behavior for example). BTW, reducing the number of lto variants checked to -O2 -flto would be fine with me.