[Bug preprocessor/69177] New: Bit-packing optimization makes it too easy to have location_t >= LINE_MAP_MAX_LOCATION_WITH_COLS

[dmalcolm at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=69177

            Bug ID: 69177
           Summary: Bit-packing optimization makes it too easy to have
                    location_t >= LINE_MAP_MAX_LOCATION_WITH_COLS
           Product: gcc
           Version: 6.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: preprocessor
          Assignee: unassigned at gcc dot gnu.org
          Reporter: dmalcolm at gcc dot gnu.org
  Target Milestone: ---

[splitting this off from PR c++/68819, and from
https://gcc.gnu.org/ml/gcc-patches/2015-12/msg01921.html, to ensure that we fix
this]

libcpp has a degraded fallback mode for large source files where if a
location_t > LINE_MAP_MAX_LOCATION_WITH_COLS we fall back to just tracking
lines, not columns&ranges (currently 0x60000000), and every location_t expands
to having a column == 0.

Sadly we're more likely to see this case in gcc 6 than in gcc 5 since we're
using up 5 bits of location_t for packing short ranges into them (to avoid
needing to go to the ad-hoc lookaside during tokenization; this change was part
of r230331).

I've seen us hit this limit on our own code during bootstrap; for example,
gimple-match.c is hitting the LINE_MAP_MAX_LOCATION_WITH_COLS limit; I've seen
a diagnostic at line 50437 column 0.

The bit-packing optimization is causing us to hit that
LINE_MAP_MAX_LOCATION_WITH_COLS limit earlier than before, and presumably this
will affect people with other very large source files (e.g. as in the initial
PR c++/68819 report).

Rough calculations, assuming 7 bits of columns,
LINE_MAP_MAX_LOCATION_WITH_COLS == 0x60000000

gcc 5:
  0x60000000 / 128 per line = 12,582,912 lines of code before
  hitting the has-column-information limit.

gcc 6 trunk:
  0x60000000 / (128 * 32) per line = 393,216 lines of code before
  hitting the has-column-information limit.

These numbers will be less in the face of lines longer than
127 characters.

I've posted a proposed fix for this as:
  https://gcc.gnu.org/ml/gcc-patches/2015-12/msg01921.html
which gives us a "longer runway", though this is mixed up with changes to
-Wmisleading-indentation