[Bug demangler/70909] Libiberty Demangler segfaults (4)

[mark at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=70909

--- Comment #53 from Mark Wielaard <mark at gcc dot gnu.org> ---
Author: mark
Date: Fri Apr 21 09:02:03 2017
New Revision: 247056

URL: https://gcc.gnu.org/viewcvs?rev=247056&root=gcc&view=rev
Log:
libiberty: Limit demangler maximum d_print_comp recursion call depth.

The fix for PR demangler/70909 and 67264 (endless demangler recursion)
catches when a demangle_component is printed in a cycle. But that doesn't
protect the call stack blowing up from non-cyclic nested types printed
recursively through d_print_comp. This can happen by a (very) long mangled
string that simply creates a very deep pointer or qualifier chain. Limit
the recursive d_print_comp call depth for a d_print_info to 1K nested
types.

libiberty/ChangeLog:

        * cp-demangle.c (MAX_RECURSION_COUNT): New constant.
        (struct d_print_info): Add recursion field.
        (d_print_init): Initialize recursion.
        (d_print_comp): Check and update d_print_info recursion depth.

Modified:
    trunk/libiberty/ChangeLog
    trunk/libiberty/cp-demangle.c