[TCWG CI] 471.omnetpp slowed down by 8% after gcc: Avoid invalid loop transformations in jump threading registry.

[Maxim Kuvyrkov]

> On 27 Sep 2021, at 19:02, Andrew MacLeod <amacleod@redhat.com> wrote:
> 
> On 9/27/21 11:39 AM, Maxim Kuvyrkov via Gcc wrote:
>>> On 27 Sep 2021, at 16:52, Aldy Hernandez <aldyh@redhat.com> wrote:
>>> 
>>> [CCing Jeff and list for broader audience]
>>> 
>>> On 9/27/21 2:53 PM, Maxim Kuvyrkov wrote:
>>>> Hi Aldy,
>>>> Your patch seems to slow down 471.omnetpp by 8% at -O3.  Could you please take a look if this is something that could be easily fixed?
>>> First of all, thanks for chasing this down.  It's incredibly useful to have these types of bug reports.
>> Thanks, Aldy, this is music to my ears :-).
>> 
>> We have built this automated benchmarking CI that bisects code-speed and code-size regressions down to a single commit.  It is still work-in-progress, and I’m forwarding these reports to patch authors, whose patches caused regressions.  If GCC community finds these useful, we can also setup posting to one of GCC’s mailing lists.
> 
> I second that this sort of thing is incredibly useful.   I don't suppose its easy to do the reverse?... let patch authors know when they've caused a significant improvement? :-)  That would be much less common I suspect, so perhaps not worth it :-)

We do this occasionally, when identifying a regression in a patch revert commit :-).  Seriously, though, it’s an easy enough code-change to the metric, but we are maxing out our benchmarking capacity with current configuration matrix.

> 
> Its certainly very useful when we are making a wholesale change to a pass which we think is beneficial, but aren't sure.
> 
> And a followup question...  Sometimes we have no good way of determining the widespread run-time effects of a change.  You seem to be running SPEC/other things continuously then?

We continuously run SPEC CPU2006 on {arm,aarch64}-{-Os/-O2/-O3}-{no LTO/LTO} matrix for GNU and LLVM toolchains.

In the GNU toolchain we track master branches and latest-release branches of Binutils, GCC and Glibc — and detect code-speed and code-size regressions across all toolchain components.

>   Does it run like once a day/some-time-period, and if you note a regression, narrow it down?

Configurations that track master branches have 3-day intervals.  Configurations that track release branches — 6 days.  If a regression is detected it is narrowed down to component first — binutils, gcc or glibc — and then the commit range of the component is bisected down to a specific commit.  All.  Done.  Automatically.

I will make a presentation on this CI at the next GNU Tools Cauldron.

>  Regardless, I think it could be very useful to be able to see the results of anything you do run at whatever frequency it happens.

Thanks!

--
Maxim Kuvyrkov
https://www.linaro.org