[Bug libstdc++/105857] codecvt::do_length causes unexpected buffer overflow

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105857

--- Comment #20 from GCC Commits <cvs-commit at gcc dot gnu.org> ---
The releases/gcc-13 branch has been updated by Jonathan Wakely
<redi@gcc.gnu.org>:

https://gcc.gnu.org/g:430a5720a97a2c725e3902fdc86d7419727a8522

commit r13-10054-g430a5720a97a2c725e3902fdc86d7419727a8522
Author: Jonathan Wakely <jwakely@redhat.com>
Date:   Tue Nov 26 20:07:33 2024 +0000

    libstdc++: Fix unsigned wraparound in codecvt::do_length [PR105857]

    When the max argument to std::codecvt<wchar_t, char, mbstate_t>::length
    is SIZE_MAX/4+1 or greater the multiplication with sizeof(wchar_t) will
    wrap to a small value, and the alloca call will have a buffer that's
    smaller than requested. The call to mbsnrtowcs then has a buffer that is
    smaller than the value passed as the buffer length. When libstdc++.so is
    built with -D_FORTIFY_SOURCE=3 the mismatched buffer and length will get
    detected and will abort inside Glibc.

    When it doesn't abort, there's no buffer overflow because Glibc's
    mbsnrtowcs has the same len * sizeof(wchar_t) calculation to determine
    the size of the buffer in bytes, and that will wrap to the same small
    number as the alloca argument. So luckily Glibc agrees with the caller
    about the real size of the buffer, and won't overflow it.

    Even when the max argument isn't large enough to wrap, it can still be
    much too large to safely pass to alloca, so we should limit that. We
    already have a loop that processes chunks so that we can handle null
    characters in the middle of the input. If we limit the alloca buffer to
    4kB then we'll just loop each time that buffer is filled.

    libstdc++-v3/ChangeLog:

            PR libstdc++/105857
            * config/locale/dragonfly/codecvt_members.cc (do_length): Limit
            size of alloca buffer to 4k.
            * config/locale/gnu/codecvt_members.cc (do_length): Likewise.
            * testsuite/22_locale/codecvt/length/wchar_t/105857.cc: New
            test.

    (cherry picked from commit 498f9aefbf36b0e3f119d634c41d86699ce6fed2)