[Bug analyzer/106147] New: RFE: -fanalyzer could complain about some cases of infinite loops and infinite recursion

[dmalcolm at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106147

            Bug ID: 106147
           Summary: RFE: -fanalyzer could complain about some cases of
                    infinite loops and infinite recursion
           Product: gcc
           Version: 12.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: analyzer
          Assignee: dmalcolm at gcc dot gnu.org
          Reporter: dmalcolm at gcc dot gnu.org
  Target Milestone: ---

We can't solve the halting problem, but maybe we can detect some cases where
the code *definitely* loops forever or has infinite recursion, where there are
no state changes or possible interactions with the outside world.

See:
  https://cwe.mitre.org/data/definitions/674.html
and:
  https://cwe.mitre.org/data/definitions/835.html

Juliet 1.3 has testcases for:
  (a) CWE674_Uncontrolled_Recursion/
  (b) CWE835_Infinite_Loop/

where (a) makes a distinction between actually unbounded vs a buggy loop that
counts down from UINT_MAX (pushing UINT_MAX stack frames is probably going to
crash).

All of the test cases in (b) perform output in an infinite loop, which I'd
argue is not a bug, as the program is generating output that's visible to the
outside world.

Compare CWE 835 examples 1 and 2.  Example 1 repeatedly calls "connect" in a
loop, which I don't think -fanalyzer is going to be able to reason about,
whereas 2 has logic:

  while (inventoryCount > minimumCount) {
    inventoryCount = inventoryCount - rateSold;
    days++;
  }

where if rateSold is 0, this effectively becomes:

  while (inventoryCount > minimumCount) {
    days++;
  }

and thus an infinite loop with no observable effects; possible interaction with
taint (e.g. if rateSold is under attacker control)