[Bug rtl-optimization/85180] Infinite (?) loop in RTL DSE optimizer

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=85180

Richard Biener <rguenth at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jakub at gcc dot gnu.org

--- Comment #2 from Richard Biener <rguenth at gcc dot gnu.org> ---
Patched find_base_term that tracks width/depth shows

#0  rhs_regno (x=0x7f3e30 <_start>)
    at /space/rguenther/src/svn/early-lto-debug/gcc/rtl.h:1895
#1  0x0000000000bb724d in find_base_term (x=0x7ffff6a84f18, width=118, 
    depth=176) at /space/rguenther/src/svn/early-lto-debug/gcc/alias.c:1894

where one major offender of width is the CSELIB loc iteration.  Adding a
hash-set to see if we recurse into already visited refs shows:

#1  0x0000000000bb7252 in find_base_term (x=0x2805d60, width=61, depth=121, 
    set=...) at /space/rguenther/src/svn/early-lto-debug/gcc/alias.c:1886
1886      gcc_assert (!set.add (x));

so indeed this is an endless recursion, not just one that is very deep.

(gdb) p debug_rtx (x)
(value:DI 11:4356 @0x2805d60/0x27f5e40)

coming via

(gdb) p debug_rtx (x)
(minus:DI (value:DI 20:20 @0x2805e38/0x27f5ff0)
    (value:DI 11:4356 @0x2805d60/0x27f5e40))

(gdb) p debug_rtx (x)
(value:DI 21:4465 @0x2805e50/0x27f6020)

(plus:DI (value:DI 21:4465 @0x2805e50/0x27f6020)
    (const_int 1 [0x1]))
...

not sure if values should ever form a cycle this way?

Anyhow - limiting search width/depth would of course catch this.