[Bug demangler/87681] New: Recursive Stack Overflow within function d_name, d_encoding, and d_local_name in cp-demangle.c, as demonstrated by "nm -C"

[N1705695H at e dot ntu.edu.sg]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87681

            Bug ID: 87681
           Summary: Recursive Stack Overflow within function d_name,
                    d_encoding, and d_local_name in cp-demangle.c, as
                    demonstrated by "nm -C"
           Product: gcc
           Version: unknown
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: demangler
          Assignee: unassigned at gcc dot gnu.org
          Reporter: N1705695H at e dot ntu.edu.sg
  Target Milestone: ---

Created attachment 44876
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=44876&action=edit
POC

An issue was discovered in cp-demangle.c in GNU libiberty, as distributed in
GNU Binutils 2.31. Stack Exhaustion occurs in the C++ demangling functions
provided by libiberty, and there is a stack consumption problem caused by
recursive stack frames: d_name, d_encoding, and d_local_name.

Please use the "./nm -C $POC" to reproduce the bug. This result can trigger
different Stack Overflow, you can try several times.


To reproduce this bug. You need to build bintuils-2.31 with ASAN. Here is the
compile Option. Another approach is to set the break Point and debug it, then
see the back trace.

> CC=clang LDFLAGS="-ldl" CFLAGS="-DFORTIFY_SOURCE=2 -fstack-protector-all -fsanitize=undefined,address -fno-omit-frame-pointer -g -O0 -Wno-error" ./configure --disable-shared --disable-gdb --disable-libdecnumber --disable-sim

The ASAN dumps the stack trace as follows:

> AddressSanitizer:DEADLYSIGNAL
> =================================================================
> ==18186==ERROR: AddressSanitizer: stack-overflow on address 0x7ffeca00cf98 (pc 0x0000008e8b7a bp 0x7ffeca00d080 sp 0x7ffeca00cfa0 T0)
>     #0 0x8e8b79 in d_name binutils-2.31/libiberty/./cp-demangle.c:1411:28
>     #1 0x8dcc1d in d_encoding binutils-2.31/libiberty/./cp-demangle.c:1312:12
>     #2 0x8ea4d3 in d_local_name binutils-2.31/libiberty/./cp-demangle.c:3590:14
>     #3 0x8e8b7e in d_name binutils-2.31/libiberty/./cp-demangle.c:1411:14
>     #4 0x8dcc1d in d_encoding binutils-2.31/libiberty/./cp-demangle.c:1312:12
>     #5 0x8ea4d3 in d_local_name binutils-2.31/libiberty/./cp-demangle.c:3590:14
>     #6 0x8e8b7e in d_name binutils-2.31/libiberty/./cp-demangle.c:1411:14
>     #7 0x8dcc1d in d_encoding binutils-2.31/libiberty/./cp-demangle.c:1312:12
>     #8 0x8ea4d3 in d_local_name binutils-2.31/libiberty/./cp-demangle.c:3590:14
>     #9 0x8e8b7e in d_name binutils-2.31/libiberty/./cp-demangle.c:1411:14
>     #10 0x8dcc1d in d_encoding binutils-2.31/libiberty/./cp-demangle.c:1312:12
>     #11 0x8ea4d3 in d_local_name binutils-2.31/libiberty/./cp-demangle.c:3590:14
>     #12 0x8e8b7e in d_name binutils-2.31/libiberty/./cp-demangle.c:1411:14
>     #13 0x8dcc1d in d_encoding binutils-2.31/libiberty/./cp-demangle.c:1312:12
>     #14 0x8ea4d3 in d_local_name binutils-2.31/libiberty/./cp-demangle.c:3590:14
>     #15 0x8e8b7e in d_name binutils-2.31/libiberty/./cp-demangle.c:1411:14
>     #16 0x8dcc1d in d_encoding binutils-2.31/libiberty/./cp-demangle.c:1312:12
>     #17 0x8ea4d3 in d_local_name binutils-2.31/libiberty/./cp-demangle.c:3590:14
>     ...
>     #246 0x8e8b7e in d_name binutils-2.31/libiberty/./cp-demangle.c:1411:14
>     #247 0x8dcc1d in d_encoding binutils-2.31/libiberty/./cp-demangle.c:1312:12
>     #248 0x8ea4d3 in d_local_name binutils-2.31/libiberty/./cp-demangle.c:3590:14
>     #249 0x8e8b7e in d_name binutils-2.31/libiberty/./cp-demangle.c:1411:14
>  SUMMARY: AddressSanitizer: stack-overflow binutils-2.31/libiberty/./cp-demangle.c:1411:28 in d_name
>  ==19901==ABORTING
>  00000000 AAborted