Bug 67264 - Infinite recursion of demangler on fuzzed input
Summary: Infinite recursion of demangler on fuzzed input
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: demangler (show other bugs)
Version: 6.0
: P3 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on: 70909
Blocks:
  Show dependency treegraph
 
Reported: 2015-08-18 23:30 UTC by Mikhail Maltsev
Modified: 2017-03-08 14:30 UTC (History)
1 user (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed:


Attachments
More testcases (13.35 KB, text/plain)
2015-08-18 23:35 UTC, Mikhail Maltsev
Details
Script for testcase reduction (3.72 KB, text/x-python)
2015-08-19 00:55 UTC, Mikhail Maltsev
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Mikhail Maltsev 2015-08-18 23:30:55 UTC
When I did fuzz-testing I had several crashes caused by infinite recursion. I did not find an easy and good solution to fix them. Perhaps, introducing some additional logic which would specifically target invalid inputs could help (like maintaining a hashtable/list/whatever of visited nodes which cannot be visited again during normal result output; substitutions would require some additional handling).
Nevertheless, I think that the testcases are still worth being recorded. Here are some, I believe, distinct examples:

_Z1KIStcvT_E - i.e. something like:
template<typename T>
K<std::operator T>
Normally, when demangling templated conversion operator we would print out the template parameter, but in this case this leads to infinite recursion, because we think that the operator itself is the parameter.
More problems with conversion operator:
_ZcvT_IIS0_EE - in substitution
_ZcvT_IZcvT_E1fE - in local name
_Z1gINcvT_EE - in nested name (probably same as std::)
_ZcvT_ILZcvDTT_EEE - template parameter in decltype

Infinite recursion when collapsing ref-qualifiers:
_Z1gIJOOT_EEOT_c

Memory hog with pointers-to-member and arrays:
_Z1KMMMMMMMMMMMMMMMA_xooooooooooooooo
(output size doubles with each M-o pair)
"pointer-to-member" and "array" are not necessarily consecutive: _ZdvMMMMMMMMMMMMMrrrrA_DTdvfp_fp_Eededilfdfdfdfd

(I first posted this as a comment for an existing bug, which was actually caused by a different issue, sorry for double-posting)
Comment 1 Mikhail Maltsev 2015-08-18 23:35:44 UTC
Created attachment 36208 [details]
More testcases
Comment 2 Mikhail Maltsev 2015-08-19 00:55:42 UTC
Created attachment 36211 [details]
Script for testcase reduction
Comment 3 Mark Wielaard 2016-12-04 23:00:15 UTC
The proposed patch for Bug 70909 - Libiberty Demangler segfaults (4) stops these kind of infinite recursion.
Comment 4 Markus Trippelsdorf 2017-03-08 14:29:10 UTC
Author: trippels
Date: Wed Mar  8 14:28:38 2017
New Revision: 245978

URL: https://gcc.gnu.org/viewcvs?rev=245978&root=gcc&view=rev
Log:
Fix PR demangler/70909 and 67264 (endless demangler recursion)

ChangeLog:

       PR demangler/70909
       PR demangler/67264
       * include/demangle.h: Add d_printing to struct demangle_component
       and pass struct demangle_component as non const.

libiberty/ChangeLog:

       PR demangler/70909
       PR demangler/67264
       * cp-demangle.c: Fix endless recursion. Pass
       struct demangle_component as non const.
       (d_make_empty): Initialize variable.
       (d_print_comp_inner): Limit recursion.
       (d_print_comp): Decrement variable.
       * cp-demint.c (cplus_demangle_fill_component): Initialize
       variable.
       (cplus_demangle_fill_builtin_type): Likewise.
       (cplus_demangle_fill_operator): Likewise.
       * testsuite/demangle-expected: Add tests.

Modified:
    trunk/ChangeLog
    trunk/include/demangle.h
    trunk/libiberty/ChangeLog
    trunk/libiberty/cp-demangle.c
    trunk/libiberty/cp-demint.c
    trunk/libiberty/testsuite/demangle-expected
Comment 5 Markus Trippelsdorf 2017-03-08 14:30:50 UTC
Fixed.