This is the mail archive of the
libstdc++@gcc.gnu.org
mailing list for the libstdc++ project.
Re: [2/2] Add AddressSanitizer annotations to std::string.
- From: Mikhail Kashkarov <m dot kashkarov at partner dot samsung dot com>
- To: Jonathan Wakely <jwakely at redhat dot com>
- Cc: "libstdc++ at gcc dot gnu dot org" <libstdc++ at gcc dot gnu dot org>, Vyacheslav Barinov <v dot barinov at samsung dot com>, Ivan Baravy <i dot baravy at samsung dot com>, gcc-patches <gcc-patches at gcc dot gnu dot org>
- Date: Fri, 8 Jun 2018 17:54:38 +0300
- Subject: Re: [2/2] Add AddressSanitizer annotations to std::string.
- Cms-type: 201P
- Dkim-filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20180608145447euoutp023d042a92df51137e0d2cb522eb95e5f4~2NxUU4Z4g2669426694euoutp02E
- References: <c8b18d2a-d5c7-e3a2-53be-30b39eadeb49@partner.samsung.com> <0d8f122d-1376-5f25-4496-0d957eebdd7b@partner.samsung.com> <CGME20180529065519eucas1p1d2e1f33524940cdc783e059622769951@eucas1p1.samsung.com> <20180529065519eucas1p1d2e1f33524940cdc783e059622769951~zCx1IxDMB0595505955eucas1p1h@eucas1p1.samsung.com> <20180529141742.GJ7974@redhat.com> <20180529151835eucas1p20ee8861f37ca38d89830d2f4aa0d8fa8~zJpPxM4Zy0176201762eucas1p2a@eucas1p2.samsung.com> <20180529155513.GM7974@redhat.com>
Hello,
I've updated patches for std::string sanitization and disabling CXX11
string SSO usage for correct sanitization.
>> _M_destroy(_M_allocated_capacity);
>>+ else
>>+ __annotate_delete();
>
>Do these calls definitely optimize away completely when not
>sanitizing? Even for -O1, -Os and -Og?
>
>For std::vector annotation I used macros to add these annotations, so
>there is no change to the generated code when annotations are
>disabled. But it makes the code quite ugly.
I've checked asm code for string-inst.o and it looks like this calls are
optimized away, but there are some light changes after patch fir .
> Right, I was wondering specifically about the <fstream>
> instantiations. I could be wrong but I don't think anything in
> <fstream> creates, destroys or modifies a std::basic_string.
I was confused by reinterpret_cast's on strings in fstream.tcc, looks
like this is not needed, you are right.
>> // calling 4.0.x+ _S_create.
>> __p->_M_set_sharable();
>>+#if _GLIBCXX_SANITIZER_ANNOTATE_STRING
>>+ __p->_M_length = 0;
>>+#endif
>
> Why is this needed? I think a comment explaining it would help (like
> the one above explaining why calling _M_set_sharable() is needed).
Checked current version without this change, looks like now it works,
reverted.
Short summary:
The reason for changing strings layout under sanitization is to place string
char buffer on heap for correct aligning in 32-bit environments,
both pre-CXX11 and CXX11 strings ABI.
| Sanitize string | string type | ABI is changed? | 32-bit | 64-bit |
|-----------------+-------------+-----------------+--------+--------|
| FULL | SSO-string | yes | + | + |
| | COW-string | yes | + | + |
|-----------------+-------------+-----------------+--------+--------|
| PARTIAL | SSO-string | no | -+(*) | + |
| | COW-string | no | - | + |
*only longs strings are sanitized for 32-bit
Some functions with new define looks a bit messy without changing internal
functions(operator=), I'm also not sure if disabling string SSO usage define
should also affects other parts that relies on basic_string class size
(checks
with static_assert in exceptions/shim-facets).
Any thoughts?
On 05/29/2018 06:55 PM, Jonathan Wakely wrote:
> On 29/05/18 18:18 +0300, Kashkarov Mikhail wrote:
>> Jonathan Wakely <jwakely@redhat.com> writes:
>>>> --- a/libstdc++-v3/include/bits/fstream.tcc
>>>> +++ b/libstdc++-v3/include/bits/fstream.tcc
>>>> @@ -1081,6 +1081,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
>>>>
>>>> // Inhibit implicit instantiations for required instantiations,
>>>> // which are defined via explicit instantiations elsewhere.
>>>> +#if !_GLIBCXX_SANITIZE_STRING
>>>> #if _GLIBCXX_EXTERN_TEMPLATE
>>>> extern template class basic_filebuf<char>;
>>>> extern template class basic_ifstream<char>;
>>>> @@ -1094,6 +1095,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
>>>> extern template class basic_fstream<wchar_t>;
>>>> #endif
>>>> #endif
>>>> +#endif // !_GLIBCXX_SANITIZE_STRING
>>>
>>> Why do we need to disable these explicit instantiation declarations?
>>> Are they affected by the std::string layout changes? Is that just
>>> because of the constructors taking std::string, or something else?
>>
>> Libstdc++ build is not sanitized, so macroses that requires
>> AddressSanitizer support will not applied and these templates will be
>> instantate without support for ASan annotations.
>
> Right, I was wondering specifically about the <fstream>
> instantiations. I could be wrong but I don't think anything in
> <fstream> creates, destroys or modifies a std::basic_string.
>
>
>
>
>
--
Best regards,
Kashkarov Mikhail
Samsung R&D Institute Russia
From 4b8de0240ac091cdd43b690276f09e94bfb0ef4d Mon Sep 17 00:00:00 2001
From: Mikhail Kashkarov <m.kashkarov@partner.samsung.com>
Date: Fri, 8 Jun 2018 14:11:11 +0300
Subject: [PATCH 2/2] Add AddressSanitizer annotations to std::string.
* include/bits/c++config: define
(_GLIBCXX_SANITIZE_STRING_PARTIAL, _GLIBCXX_SANITIZE_STRING_FULL)
(_GLIBCXX_SANTIZE_STRING, _GLIBCXX_SANITIZER_ANNOTATE_STRING)
(_GLIBCXX_SANITIZER_DISABLE_LOCAL_STRING_ANNOTATION)
(_GLIBCXX_SANITIZER_ALIGN_COW_STRING)
* doc/xml/manual/using.xml: document GLIBCXX_SANITIZE_STRING_PARTIAL,
_GLIBCXX_SANITIZE_STRING_FULL
* include/bits/basic_string.h [_GLIBCXX_USE_DUAL_ABI]
(_asan_traits<_CharT, _Alloc>, _asan_traits<_CharT, allocator<_CharT>)
(_asan_traits::__annotate_delete, _asan_traits::__annotate_new)
(_asan_traits::__annotate_grow): New traits for annotation.
(basic_string::__RAII_IncreaseAnnotator::__RAII_Increaseannotator)
(basic_string::__RAII_IncreaseAnnotator::done)
(basic_string::__RAII_IncreaseAnnotator::~RAII_Increaseannotator):
New annotation helpers in case of exceptions.
(basic_string::__get_beg, basic_string::__get_mid)
(basic_string::__get_end): New annotation helpers.
(basis_string::_M_dispose, basic_string::_M_destroy)
(basic_string::basic_string(basic_string&& __str))
(basic_string::basic_string(basic_string&& __str, const _Alloc& __a))
(basic_string::operator=(constbasic_string& __str))
(basic_string::operator=(basic_string&& __str))
(basic_string::clear, basic_string::push_back): Annotate.
[!_GLIBCXX_USE_DUAL_ABI]
(_asan_traits<_CharT, _Alloc>, _asan_traits<_CharT, allocator<_CharT>)
(_asan_traits::__annotate_delete, _asan_traits::__annotate_new)
(_asan_traits::__annotate_grow): New traits for annotation.
(basic_string::__RAII_IncreaseAnnotator::__RAII_Increaseannotator)
(basic_string::__RAII_IncreaseAnnotator::done)
(basic_string::__RAII_IncreaseAnnotator::~RAII_Increaseannotator):
New annotation helpers in case of exceptions.
(basic_string::__get_beg, basic_string::__get_mid)
(basic_string::__get_end): New annotation helpers.
(basic_string::_Rep_base): Align by 8 when annotations are anabled.
(basic_string::push_back): Annotate
* include/bits/basic_string.tcc [_GLIBCXX_USE_DUAL_ABI]
(basic_string::swap, basic_string::_M_construct)
(basic_string::_M_assign, basic_string::reserve)
(basic_string::_M_mutate, basic_string::_M_erase)
(basic_string::resize, basic_string::_M_append)
(basic_string::_M_replace): Annotate.
[!_GLIBCXX_USE_DUAL_ABI]
(basic_string::_S_construct)
(basic_string::append(const _CharT* __s, size_type __n))
(basic_string::append(const basic_string& __str))
(basic_string::_M_destroy, basic_string::_M_mutate)
(basic_string::_S_create, basic_string::_M_clone): Annotate.
Disable template implicit instantiation declarations for annotations.
* libstdc++-v3/include/bits/sstream.tcc
(basic_stringbuf::overflow, basic_stringbuf::_M_sync): Annotate
Disable template implicit instantiation declarations for annotations.
* libstdc++-v3/include/bits/locale_facets.tcc:
Disable template implicit instantiation declarations for annotations.
* libstdc++-v3/include/bits/fstream.tcc: Likewise.
---
libstdc++-v3/doc/xml/manual/using.xml | 41 ++++
libstdc++-v3/include/bits/basic_string.h | 314 +++++++++++++++++++++++++++-
libstdc++-v3/include/bits/basic_string.tcc | 66 +++++-
libstdc++-v3/include/bits/c++config | 28 +++
libstdc++-v3/include/bits/locale_facets.tcc | 2 +
libstdc++-v3/include/bits/sstream.tcc | 8 +
6 files changed, 450 insertions(+), 9 deletions(-)
diff --git a/libstdc++-v3/doc/xml/manual/using.xml b/libstdc++-v3/doc/xml/manual/using.xml
index 67f9cf5..ba5af0d 100644
--- a/libstdc++-v3/doc/xml/manual/using.xml
+++ b/libstdc++-v3/doc/xml/manual/using.xml
@@ -992,6 +992,47 @@ g++ -Winvalid-pch -I. -include stdc++.h -H -g -O2 hello.cc -o test.exe
</para>
</listitem></varlistentry>
+ <varlistentry><term><code>_GLIBCXX_SANITIZE_STRING_PARTIAL</code></term>
+ <listitem>
+ <para>
+ Undefined by default. When defined, <classname>std::string</classname>
+ operations will be annotated so that AddressSanitizer can detect
+ invalid accesses to the unused capacity of a
+ <classname>std::string</classname>. These annotations are only
+ enabled for
+ <classname>std::basic_string<T, std::allocator<T>></classname>
+ and only when <classname>std::allocator</classname> is derived from
+ <xref linkend="allocator.impl"><classname>new_allocator</classname>
+ or <classname>malloc_allocator</classname></xref>. The annotations
+ must be present on all string operations or none, so this macro must be
+ defined to the same value for all translation units that create, destroy or
+ modify strings. With this partial sanitization CXX11 small strings wiil be not
+ annotated, 64-bit environment is required for COW-strings to align
+ character buffer correctly.
+ </para>
+ </listitem></varlistentry>
+
+ <varlistentry><term><code>_GLIBCXX_SANITIZE_STRING_FULL</code></term>
+ <listitem>
+ <para>
+ Undefined by default. When defined, <classname>std::string</classname>
+ operations will be annotated so that AddressSanitizer can detect
+ invalid accesses to the unused capacity of a
+ <classname>std::string</classname>. These annotations are only
+ enabled for
+ <classname>std::basic_string<T, std::allocator<T>></classname>
+ and only when <classname>std::allocator</classname> is derived from
+ <xref linkend="allocator.impl"><classname>new_allocator</classname>
+ or <classname>malloc_allocator</classname></xref>. The annotations
+ must be present on all string operations or none, so this macro must
+ be defined to the same value for all translation units that create,
+ destroy or modify strings. Strings ABI is changed: pre-CXX11 string
+ character buffer is forced to be aligned by 8 with annotations, disable
+ small-string optimization for CXX11 stings to allow correct AddressSanitizer
+ poisoning.
+ </para>
+ </listitem></varlistentry>
+
<varlistentry><term><code>_GLIBCXX_SANITIZE_VECTOR</code></term>
<listitem>
<para>
diff --git a/libstdc++-v3/include/bits/basic_string.h b/libstdc++-v3/include/bits/basic_string.h
index 9d971c0..d0448d9 100644
--- a/libstdc++-v3/include/bits/basic_string.h
+++ b/libstdc++-v3/include/bits/basic_string.h
@@ -48,11 +48,84 @@
# include <string_view>
#endif
+#if _GLIBCXX_SANITIZER_ANNOTATE_STRING
+extern "C" void
+__sanitizer_annotate_contiguous_container(const void *, const void *,
+ const void *, const void *);
+#endif
namespace std _GLIBCXX_VISIBILITY(default)
{
_GLIBCXX_BEGIN_NAMESPACE_VERSION
+ template <typename _CharT, typename _Alloc>
+ struct _Asan_traits
+ {
+ typedef typename
+ __gnu_cxx::__alloc_traits<_Alloc>::const_pointer const_pointer;
+
+ static void __annotate_delete(const_pointer __beg,
+ const_pointer __mid,
+ const_pointer __end) { }
+
+ static void __annotate_new(const_pointer __beg, const_pointer __mid,
+ const_pointer __end) { }
+
+ static void __annotate_grow(const_pointer __beg,
+ const_pointer __old_mid,
+ const_pointer __new_mid,
+ const_pointer __end) { }
+ };
+
+ // AddressSanitizer is enabled only for default allocator.
+#if _GLIBCXX_SANITIZER_ANNOTATE_STRING
+ template <typename _CharT>
+ struct _Asan_traits<_CharT, allocator<_CharT> >
+ {
+ typedef typename
+ __gnu_cxx::__alloc_traits<allocator_CharT>::const_pointer const_pointer;
+
+ static void _GLIBCXX_VISIBILITY(hidden) __annotate_delete(
+ const_pointer __beg,
+ const_pointer __mid,
+ const_pointer __end)
+ {
+ __annotate_contiguous_container(__beg, __end, __mid, __end);
+ }
+
+ static void _GLIBCXX_VISIBILITY(hidden) __annotate_new(
+ const_pointer __beg,
+ const_pointer __mid,
+ const_pointer __end)
+ {
+ __annotate_contiguous_container(__beg, __end, __end, __mid);
+ }
+
+ static void _GLIBCXX_VISIBILITY(hidden) __annotate_grow(
+ const_pointer __beg,
+ const_pointer __old_mid,
+ const_pointer __new_mid,
+ const_pointer __end)
+ {
+ __annotate_contiguous_container(__beg, __end, __old_mid, __new_mid);
+ }
+
+ private:
+ static void _GLIBCXX_VISIBILITY(hidden) __annotate_contiguous_container(
+ const void* __beg,
+ const void* __end,
+ const void* __old_mid,
+ const void* __new_mid)
+ {
+ if (__beg)
+ {
+ __sanitizer_annotate_contiguous_container(__beg, __end, __old_mid,
+ __new_mid);
+ }
+ }
+ };
+#endif // _GLIBCXX_SANITIZER_ANNOTATE_STRING
+
#if _GLIBCXX_USE_CXX11_ABI
_GLIBCXX_BEGIN_NAMESPACE_CXX11
/**
@@ -96,11 +169,130 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
const_iterator;
typedef std::reverse_iterator<const_iterator> const_reverse_iterator;
typedef std::reverse_iterator<iterator> reverse_iterator;
+ typedef _Asan_traits<_CharT, _Char_alloc_type> asan_traits;
/// Value returned by various member functions when they fail.
static const size_type npos = static_cast<size_type>(-1);
private:
+#if _GLIBCXX_SANITIZER_ANNOTATE_STRING
+ // The annotation for size increase should happen before the actual
+ // increase, but if an exception is thrown after that the annotation has
+ // to be undone.
+ struct _GLIBCXX_VISIBILITY(hidden) _RAII_IncreaseAnnotator
+ {
+ _RAII_IncreaseAnnotator (const basic_string& __s,
+ difference_type __n = 1)
+ : __commit(false), __s(__s), __n(__n)
+ {
+ if (__n > 0) // grow before use
+ {
+ size_type __old_size = __s.size();
+ if (__s._M_is_local()
+ && ((__s.size() + __n) > __s.capacity()))
+ {
+ __n = __s.capacity() - __s.size();
+ }
+ __s.__annotate_grow(__old_size, __old_size + __n);
+ }
+ }
+
+ void __done()
+ {
+ if (__n < 0) // shrink after use
+ {
+ size_type __old_size = __s.size();
+ __s.__annotate_grow(__old_size, __old_size + __n);
+ }
+ __commit = true;
+ }
+
+ ~_RAII_IncreaseAnnotator()
+ {
+ if (__commit) return;
+ size_type __cur_size = __s.size();
+ if (__s._M_is_local())
+ __s.__annotate_grow(__s.capacity(), __cur_size);
+ else
+ __s.__annotate_grow(__cur_size + __n, __cur_size);
+ }
+
+ bool __commit;
+ difference_type __n;
+ const basic_string &__s;
+ };
+#else // _GLIBCXX_SANITIZER_ANNOTATE_STRING
+ struct _GLIBCXX_VISIBILITY(hidden) _RAII_IncreaseAnnotator
+ {
+ inline _RAII_IncreaseAnnotator(const basic_string &, size_type __n = 1)
+ {}
+ inline void __done() { }
+ ~_RAII_IncreaseAnnotator() { }
+ };
+#endif // _GLIBCXX_SANITIZER_ANNOTATE_STRING
+
+ const_pointer _S_get_beg () const _GLIBCXX_VISIBILITY(hidden)
+ {
+ if (!_M_is_local())
+ return _M_data();
+ return _M_local_data();
+ }
+
+ const_pointer _S_get_mid(size_type __sz) const _GLIBCXX_VISIBILITY(hidden)
+ {
+ if (!_M_is_local())
+ return _M_data() + __sz + 1 /* terminator */;
+ else
+ return _M_local_data() + __sz + 1;
+ }
+
+ const_pointer _S_get_end() const _GLIBCXX_VISIBILITY(hidden)
+ {
+ if (!_M_is_local())
+ return _M_data() + _M_allocated_capacity + 1;
+
+ const_pointer __p = _S_get_beg() + _S_local_capacity + 1;
+
+ return __p;
+ }
+
+ void __annotate_new(size_type __sz) const _GLIBCXX_VISIBILITY(hidden)
+ {
+#ifdef _GLIBCXX_SANITIZER_DISABLE_LOCAL_STRING_ANNOTATION
+ if (_M_is_local())
+ return;
+#endif
+ asan_traits::__annotate_new(_S_get_beg(), _S_get_mid(__sz),
+ _S_get_end());
+ }
+
+ void __annotate_grow(size_type __old_size, size_type __new_size) const
+ _GLIBCXX_VISIBILITY(hidden)
+ {
+#ifdef _GLIBCXX_SANITIZER_DISABLE_LOCAL_STRING_ANNOTATION
+ if (_M_is_local())
+ return;
+#endif
+ asan_traits::__annotate_grow(_S_get_beg(), _S_get_mid(__old_size),
+ _S_get_mid(__new_size), _S_get_end());
+ }
+
+ basic_string<_CharT, _Traits, _Alloc> const *__annotate_delete() const
+ _GLIBCXX_VISIBILITY(hidden)
+ {
+#ifdef _GLIBCXX_SANITIZER_DISABLE_LOCAL_STRING_ANNOTATION
+ if (_M_is_local())
+ return this;
+#endif
+ if (_M_is_local())
+ asan_traits::__annotate_delete(_S_get_beg(), _S_get_mid(0),
+ _S_get_end());
+ else
+ asan_traits::__annotate_delete(_S_get_beg(), _S_get_mid(size()),
+ _S_get_end());
+ return this;
+ }
+
// type used for positions in insert, erase etc.
#if __cplusplus < 201103L
typedef iterator __const_iterator;
@@ -225,6 +417,8 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
{
if (!_M_is_local())
_M_destroy(_M_allocated_capacity);
+ else
+ __annotate_delete();
}
void
@@ -234,6 +428,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
if (!_M_allocated_capacity)
return;
#endif
+ __annotate_delete();
_Alloc_traits::deallocate(_M_get_allocator(), _M_data(), __size + 1);
}
@@ -560,8 +755,10 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
#if !_GLIBCXX_DISABLE_STRING_SSO_USAGE
if (__str._M_is_local())
{
+ __str.__annotate_delete();
traits_type::copy(_M_local_buf, __str._M_local_buf,
_S_local_capacity + 1);
+ __annotate_new(__str.length());
}
else
#endif
@@ -582,6 +779,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
__str._M_data(__str._M_local_data());
__str._M_set_length(0);
#endif
+ __str.__annotate_new(0);
}
/**
@@ -607,6 +805,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
traits_type::copy(_M_local_buf, __str._M_local_buf,
_S_local_capacity + 1);
_M_length(__str.length());
+ __annotate_new(__str.length());
__str._M_set_length(0);
}
else
@@ -617,6 +816,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
_M_data(__str._M_data());
_M_length(__str.length());
_M_capacity(__str._M_allocated_capacity);
+ __annotate_new(__str.length());
#if _GLIBCXX_DISABLE_STRING_SSO_USAGE
__str._M_data(nullptr);
__str._M_length(0);
@@ -625,7 +825,6 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
__str._M_data(__str._M_local_buf);
__str._M_set_length(0);
#endif
-
}
else
_M_construct(__str.begin(), __str.end());
@@ -729,6 +928,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
_M_data(__ptr);
_M_capacity(__new_cap);
_M_set_length(__len);
+ __annotate_new(__len);
}
}
std::__alloc_on_copy(_M_get_allocator(), __str._M_get_allocator());
@@ -774,6 +974,10 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
operator=(basic_string&& __str)
noexcept(_Alloc_traits::_S_nothrow_move())
{
+#if _GLIBCXX_SANITIZER_ANNOTATE_STRING
+ size_type __str_capacity = __str.capacity();
+ size_type __str_length = __str.length();
+#endif
if (!_M_is_local() && _Alloc_traits::_S_propagate_on_move_assign()
&& !_Alloc_traits::_S_always_equal()
&& _M_get_allocator() != __str._M_get_allocator())
@@ -786,6 +990,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
#else
_M_data(_M_local_data());
_M_set_length(0);
+ __annotate_new(0);
#endif
}
// Replace allocator if POCMA is true.
@@ -797,16 +1002,18 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
{
pointer __data = nullptr;
size_type __capacity;
-#if _GLIBCXX_DISABLE_STRING_SSO_USAGE
+#if _GLIBCXX_DISABLE_STRING_SSO_USAGE || _GLIBCXX_SANITIZER_ANNOTATE_STRING
size_type __length;
#endif
+ __annotate_delete();
+ __str.__annotate_delete();
if (!_M_is_local())
{
if (_Alloc_traits::_S_always_equal())
{
__data = _M_data();
__capacity = _M_allocated_capacity;
-#if _GLIBCXX_DISABLE_STRING_SSO_USAGE
+#if _GLIBCXX_DISABLE_STRING_SSO_USAGE || _GLIBCXX_SANITIZER_ANNOTATE_STRING
__length = _M_string_length;
#endif
}
@@ -826,8 +1033,9 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
{
__str._M_data(__data);
__str._M_capacity(__capacity);
-#if _GLIBCXX_DISABLE_STRING_SSO_USAGE
+#if _GLIBCXX_DISABLE_STRING_SSO_USAGE || _GLIBCXX_SANITIZER_ANNOTATE_STRING
__str._M_length(__length);
+ __str.__annotate_new(__str.length());
#endif
}
else {
@@ -837,8 +1045,10 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
__str._M_capacity(0);
#else
__str._M_data(__str._M_local_buf);
+ __str.__annotate_new(__str.length());
#endif
}
+ __annotate_new(length());
}
else
assign(__str);
@@ -1081,6 +1291,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
void
clear() _GLIBCXX_NOEXCEPT
{
+ __annotate_grow(_M_string_length, 0);
#if _GLIBCXX_DISABLE_STRING_SSO_USAGE
_M_allocated_capacity ? _M_set_length(0) : _M_length(0);
#else
@@ -1416,6 +1627,7 @@ _GLIBCXX_BEGIN_NAMESPACE_CXX11
const size_type __size = this->size();
if (__size + 1 > this->capacity())
this->_M_mutate(__size, size_type(0), 0, size_type(1));
+ this->__annotate_grow(__size, __size + 1);
traits_type::assign(this->_M_data()[__size], __c);
this->_M_set_length(__size + 1);
}
@@ -3208,12 +3420,57 @@ _GLIBCXX_END_NAMESPACE_CXX11
size_type _M_capacity;
_Atomic_word _M_refcount;
};
+ // } _GLIBCXX_SANITIZER_ALIGN_COW_STRING;
struct _Rep : _Rep_base
{
// Types:
typedef typename _Alloc::template rebind<char>::other _Raw_bytes_alloc;
+ typedef _Asan_traits<_CharT, _CharT_alloc_type> asan_traits;
+ #if _GLIBCXX_SANITIZE_ANNOTATE_STRING
+ const_pointer _S_get_beg() const _GLIBCXX_NOEXCEPT
+ {
+ return _M_refdata();
+ }
+
+ const_pointer _S_get_mid(size_type __sz) const _GLIBCXX_NOEXCEPT
+ {
+ return std::min(_M_refdata() + __sz + 1, /* terminator */
+ _S_get_end());
+ }
+ const_pointer _S_get_end() const _GLIBCXX_NOEXCEPT
+ {
+ return _M_refdata() + this->_M_capacity + 1 /* terminator */;
+ }
+
+ void __annotate_new(size_type __sz) const
+ {
+ asan_traits::__annotate_new(_S_get_beg(), _S_get_mid(__sz),
+ _S_get_end());
+ }
+
+ void __annotate_grow(size_type __old_size, size_type __new_size) const
+ {
+ asan_traits::__annotate_grow(_S_get_beg(), _S_get_mid(__old_size),
+ _S_get_mid(__new_size), _S_get_end());
+ }
+
+ _Rep const *__annotate_delete() const
+ {
+ asan_traits::__annotate_delete(_S_get_beg(),
+ _S_get_mid(this->_M_length),
+ _S_get_end());
+ return this;
+ }
+#else // _GLIBCXX_SANITIZE_ANNOTATE_STRING
+ const_pointer _S_get_beg() const _GLIBCXX_NOEXCEPT { return 0; }
+ const_pointer _S_get_mid(size_type __sz) const _GLIBCXX_NOEXCEPT { return 0; }
+ const_pointer _S_get_end() const _GLIBCXX_NOEXCEPT { return 0; }
+ void __annotate_new(size_type __sz) const { }
+ void __annotate_grow(size_type __old_size, size_type __new_size) const { }
+ _Rep const *__annotate_delete() const { return this; }
+#endif // _GLIBCXX_SANITIZE_ANNOTATE_STRING
// (Public) Data members:
// The maximum number of individual char_type elements of an
@@ -3296,6 +3553,10 @@ _GLIBCXX_END_NAMESPACE_CXX11
}
}
+ const_pointer
+ _M_refdata() const throw()
+ { return reinterpret_cast<const_pointer>(this + 1); }
+
_CharT*
_M_refdata() throw()
{ return reinterpret_cast<_CharT*>(this + 1); }
@@ -3354,6 +3615,50 @@ _GLIBCXX_END_NAMESPACE_CXX11
_M_clone(const _Alloc&, size_type __res = 0);
};
+#if _GLIBCXX_SANITIZER_ANNOTATE_STRING
+ // The annotation for size increase should happen before the actual
+ // increase, but if an exception is thrown after that the annotation has
+ // to be undone.
+ struct _GLIBCXX_VISIBILITY(hidden) _RAII_IncreaseAnnotator
+ {
+ _RAII_IncreaseAnnotator (const _Rep& __s, difference_type __n = 1)
+ : __commit(false), __s(__s), __n(__n)
+ {
+ if (__n > 0) // grow before use
+ {
+ size_type __old_size = __s._M_length;
+ __s.__annotate_grow(__old_size, __old_size + __n);
+ }
+ }
+ void __done()
+ {
+ if (__n < 0) // shrink after use
+ {
+ size_type __old_size = __s._M_length;
+ __s.__annotate_grow(__old_size, __old_size + __n);
+ }
+ __commit = true;
+ }
+ ~_RAII_IncreaseAnnotator()
+ {
+ if (__commit) return;
+ size_type __cur_size = __s._M_length;
+ __s.__annotate_grow(__cur_size + __n, __cur_size);
+ }
+ bool __commit;
+ difference_type __n;
+ const _Rep &__s;
+ };
+#else // _GLIBCXX_SANITIZER_ANNOTATE_STRING
+ struct _GLIBCXX_VISIBILITY(hidden) _RAII_IncreaseAnnotator
+ {
+ inline _RAII_IncreaseAnnotator(const _Rep &, size_type __n = 1)
+ {}
+ inline void __done() {}
+ ~_RAII_IncreaseAnnotator() {}
+ };
+#endif // _GLIBCXX_SANITIZER_ANNOTATE_STRING
+
// Use empty-base optimization: http://www.cantrip.org/emptyopt.html
struct _Alloc_hider : _Alloc
{
@@ -4318,6 +4623,7 @@ _GLIBCXX_END_NAMESPACE_CXX11
push_back(_CharT __c)
{
const size_type __len = 1 + this->size();
+ _M_rep()->__annotate_grow(this->size(), this->size() + 1);
if (__len > this->capacity() || _M_rep()->_M_is_shared())
this->reserve(__len);
traits_type::assign(_M_data()[this->size()], __c);
diff --git a/libstdc++-v3/include/bits/basic_string.tcc b/libstdc++-v3/include/bits/basic_string.tcc
index 6b6c7eb..89d57d3 100644
--- a/libstdc++-v3/include/bits/basic_string.tcc
+++ b/libstdc++-v3/include/bits/basic_string.tcc
@@ -62,6 +62,8 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
return;
_Alloc_traits::_S_on_swap(_M_get_allocator(), __s._M_get_allocator());
+ __annotate_delete();
+ __s.__annotate_delete();
if (_M_is_local())
if (__s._M_is_local())
@@ -124,7 +126,9 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
const size_type __tmp_length = length();
_M_length(__s.length());
+ __annotate_new(length());
__s._M_length(__tmp_length);
+ __s.__annotate_new(__s.length());
}
template<typename _CharT, typename _Traits, typename _Alloc>
@@ -175,8 +179,10 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
while (__beg != __end && __len < __capacity)
{
+ _RAII_IncreaseAnnotator __annotator(*this);
_M_data()[__len++] = *__beg;
++__beg;
+ __annotator.__done();
}
__try
@@ -196,9 +202,12 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
_M_dispose();
_M_data(__another);
_M_capacity(__capacity);
+ __annotate_new(__len);
}
+ _RAII_IncreaseAnnotator __annotator(*this);
_M_data()[__len++] = *__beg;
++__beg;
+ __annotator.__done();
}
}
__catch(...)
@@ -235,6 +244,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
{
_M_data(_M_create(__new_capacity, size_type(0)));
_M_capacity(__new_capacity);
+ __annotate_new(__dnew);
}
// Check for out_of_range and length_error exceptions.
@@ -266,6 +276,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
{
_M_data(_M_create(__new_capacity, size_type(0)));
_M_capacity(__new_capacity);
+ __annotate_new(__n);
}
if (__n)
@@ -281,6 +292,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
{
if (this != &__str)
{
+ __annotate_delete();
const size_type __rsize = __str.length();
const size_type __capacity = capacity();
@@ -301,6 +313,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
#else
_M_set_length(__rsize);
#endif
+ __annotate_new(__rsize);
}
}
@@ -335,6 +348,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
_M_data(_M_local_data());
}
#endif
+ __annotate_new(length());
}
}
@@ -360,6 +374,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
_M_dispose();
_M_data(__r);
_M_capacity(__new_capacity);
+ __annotate_new(__pos + __len2 + __how_much);
}
template<typename _CharT, typename _Traits, typename _Alloc>
@@ -372,7 +387,9 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
if (__how_much && __n)
this->_S_move(_M_data() + __pos, _M_data() + __pos + __n, __how_much);
+ const size_type __old_length = length();
_M_set_length(length() - __n);
+ __annotate_grow(__old_length, __old_length - __n);
}
template<typename _CharT, typename _Traits, typename _Alloc>
@@ -384,7 +401,10 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
if (__size < __n)
this->append(__n - __size, __c);
else if (__n < __size)
- this->_M_set_length(__n);
+ {
+ this->_M_set_length(__n);
+ this->__annotate_grow(__size, __n);
+ }
}
template<typename _CharT, typename _Traits, typename _Alloc>
@@ -392,12 +412,16 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
basic_string<_CharT, _Traits, _Alloc>::
_M_append(const _CharT* __s, size_type __n)
{
- const size_type __len = __n + this->size();
+ const size_type __old_len = this->size();
+ const size_type __len = __n + __old_len;
if (__len <= this->capacity())
{
if (__n)
- this->_S_copy(this->_M_data() + this->size(), __s, __n);
+ {
+ __annotate_grow(__old_len, __len);
+ this->_S_copy(this->_M_data() + __old_len, __s, __n);
+ }
}
else
this->_M_mutate(this->size(), size_type(0), __s, __n);
@@ -436,8 +460,10 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
pointer __p = this->_M_data() + __pos1;
const size_type __how_much = __old_size - __pos1 - __n1;
+ _RAII_IncreaseAnnotator __annotator(*this, __n2 - __n1);
if (__how_much && __n1 != __n2)
this->_S_move(__p + __n2, __p + __n1, __how_much);
+ __annotator.__done();
}
else
this->_M_mutate(__pos1, __n1, 0, __n2);
@@ -462,6 +488,9 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
if (__new_size <= this->capacity())
{
+ // grow before use
+ if (__old_size < __new_size)
+ __annotate_grow(__old_size, __new_size);
pointer __p = this->_M_data() + __pos;
const size_type __how_much = __old_size - __pos - __len1;
@@ -494,6 +523,10 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
}
}
}
+ // shrink after use
+ if (__old_size > __new_size) {
+ __annotate_grow(__old_size, __new_size);
+ }
}
else
this->_M_mutate(__pos, __len1, __s, __len2);
@@ -591,6 +624,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
__r->_M_destroy(__a);
__throw_exception_again;
}
+ __r->__annotate_new(__len);
__r->_M_set_length_and_sharable(__len);
return __r->_M_refdata();
}
@@ -621,6 +655,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
__r->_M_destroy(__a);
__throw_exception_again;
}
+ __r->__annotate_new(__dnew);
__r->_M_set_length_and_sharable(__dnew);
return __r->_M_refdata();
}
@@ -636,6 +671,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
#endif
// Check for out_of_range and length_error exceptions.
_Rep* __r = _Rep::_S_create(__n, size_type(0), __a);
+ __r->__annotate_new(__n);
if (__n)
_M_assign(__r->_M_refdata(), __n, __c);
@@ -771,10 +807,13 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
if (__n)
{
_M_check_length(size_type(0), __n, "basic_string::append");
- const size_type __len = __n + this->size();
+ const size_type __old_len = this->size();
+ const size_type __len = __n + __old_len;
if (__len > this->capacity() || _M_rep()->_M_is_shared())
this->reserve(__len);
+ _RAII_IncreaseAnnotator __annotator(*_M_rep(), __n);
_M_assign(_M_data() + this->size(), __n, __c);
+ __annotator.__done();
_M_rep()->_M_set_length_and_sharable(__len);
}
return *this;
@@ -801,6 +840,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
__s = _M_data() + __off;
}
}
+ _M_rep()->__annotate_grow(this->size(), __len);
_M_copy(_M_data() + this->size(), __s, __n);
_M_rep()->_M_set_length_and_sharable(__len);
}
@@ -815,9 +855,11 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
const size_type __size = __str.size();
if (__size)
{
- const size_type __len = __size + this->size();
+ const size_type __old_len = this->size();
+ const size_type __len = __size + __old_len;
if (__len > this->capacity() || _M_rep()->_M_is_shared())
this->reserve(__len);
+ _M_rep()->__annotate_grow(__old_len, __len);
_M_copy(_M_data() + this->size(), __str._M_data(), __size);
_M_rep()->_M_set_length_and_sharable(__len);
}
@@ -934,6 +976,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
{
const size_type __size = sizeof(_Rep_base) +
(this->_M_capacity + 1) * sizeof(_CharT);
+ __annotate_delete();
_Raw_bytes_alloc(__a).deallocate(reinterpret_cast<char*>(this), __size);
}
@@ -965,12 +1008,16 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
// Must reallocate.
const allocator_type __a = get_allocator();
_Rep* __r = _Rep::_S_create(__new_size, this->capacity(), __a);
+ __r->_M_length = 0;
+ __r->__annotate_new(0);
+ _RAII_IncreaseAnnotator __annotator(*__r, __new_size);
if (__pos)
_M_copy(__r->_M_refdata(), _M_data(), __pos);
if (__how_much)
_M_copy(__r->_M_refdata() + __pos + __len2,
_M_data() + __pos + __len1, __how_much);
+ __annotator.__done();
_M_rep()->_M_dispose(__a);
_M_data(__r->_M_refdata());
@@ -978,9 +1025,15 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
else if (__how_much && __len1 != __len2)
{
// Work in-place.
+ if (__new_size > __old_size)
+ _M_rep()->__annotate_grow(__old_size, __new_size);
_M_move(_M_data() + __pos + __len2,
_M_data() + __pos + __len1, __how_much);
+ if (__new_size < __old_size)
+ _M_rep()->__annotate_grow(__old_size, __new_size);
}
+ else
+ _M_rep()->__annotate_grow(__old_size, __new_size);
_M_rep()->_M_set_length_and_sharable(__new_size);
}
@@ -1115,6 +1168,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
const size_type __requested_cap = this->_M_length + __res;
_Rep* __r = _Rep::_S_create(__requested_cap, this->_M_capacity,
__alloc);
+ __r->__annotate_new(this->_M_length);
if (this->_M_length)
_M_copy(__r->_M_refdata(), _M_refdata(), this->_M_length);
@@ -1635,6 +1689,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
// Inhibit implicit instantiations for required instantiations,
// which are defined via explicit instantiations elsewhere.
+#if !_GLIBCXX_SANITIZE_STRING
#if _GLIBCXX_EXTERN_TEMPLATE > 0 && __cplusplus <= 201402L
extern template class basic_string<char>;
extern template
@@ -1666,6 +1721,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
getline(basic_istream<wchar_t>&, wstring&);
#endif
#endif
+#endif
_GLIBCXX_END_NAMESPACE_VERSION
} // namespace std
diff --git a/libstdc++-v3/include/bits/c++config b/libstdc++-v3/include/bits/c++config
index 280f65e..0f29f9c 100644
--- a/libstdc++-v3/include/bits/c++config
+++ b/libstdc++-v3/include/bits/c++config
@@ -420,6 +420,34 @@ namespace std
# define _GLIBCXX_END_NAMESPACE_LDBL_OR_CXX11 _GLIBCXX_END_NAMESPACE_LDBL
#endif
+// Partial AddressSanitizer std::basic_string annotation:
+// ABI is unchanged, short strings are not sanitized,
+// works with cow-strings only under 64-bits environment.
+#if _GLIBCXX_SANITIZE_STRING_PARTIAL
+# define _GLIBCXX_SANITIZE_STRING 1
+# define _GLIBCXX_DISABLE_STRING_SSO_USAGE 0
+# define _GLIBCXX_SANITIZER_ALIGN_COW_STRING
+# if _GLIBCXX_SANITIZE_STD_ALLOCATOR
+# define _GLIBCXX_SANITIZER_ANNOTATE_STRING 1
+# define _GLIBCXX_SANITIZER_DISABLE_LOCAL_STRING_ANNOTATION
+# endif
+#endif
+
+// Full AddressSanitizer std::basic_string annotation: ABI is changed.
+#if _GLIBCXX_SANITIZE_STRING_FULL
+# define _GLIBCXX_SANITIZE_STRING 1
+# define _GLIBCXX_DISABLE_STRING_SSO_USAGE 1
+# define _GLIBCXX_SANITIZER_ALIGN_COW_STRING __attribute__((aligned(8)))
+# if _GLIBCXX_SANITIZE_STD_ALLOCATOR
+# define _GLIBCXX_SANITIZER_ANNOTATE_STRING 1
+# undef _GLIBCXX_SANITIZER_DISABLE_LOCAL_STRING_ANNOTATION
+# endif
+#endif
+
+#if !_GLIBCXX_SANITIZE_STRING
+# define _GLIBCXX_SANITIZER_ALIGN_COW_STRING
+#endif
+
// Debug Mode implies checking assertions.
#if defined(_GLIBCXX_DEBUG) && !defined(_GLIBCXX_ASSERTIONS)
# define _GLIBCXX_ASSERTIONS 1
diff --git a/libstdc++-v3/include/bits/locale_facets.tcc b/libstdc++-v3/include/bits/locale_facets.tcc
index 39da5766..31ee6a4 100644
--- a/libstdc++-v3/include/bits/locale_facets.tcc
+++ b/libstdc++-v3/include/bits/locale_facets.tcc
@@ -1291,6 +1291,7 @@ _GLIBCXX_END_NAMESPACE_LDBL
// Inhibit implicit instantiations for required instantiations,
// which are defined via explicit instantiations elsewhere.
+#if !_GLIBCXX_SANITIZE_STRING
#if _GLIBCXX_EXTERN_TEMPLATE
extern template class _GLIBCXX_NAMESPACE_CXX11 numpunct<char>;
extern template class _GLIBCXX_NAMESPACE_CXX11 numpunct_byname<char>;
@@ -1370,6 +1371,7 @@ _GLIBCXX_END_NAMESPACE_LDBL
has_facet<num_get<wchar_t> >(const locale&);
#endif
#endif
+#endif // !_GLIBCXX_SANITIZE_STRING
_GLIBCXX_END_NAMESPACE_VERSION
} // namespace
diff --git a/libstdc++-v3/include/bits/sstream.tcc b/libstdc++-v3/include/bits/sstream.tcc
index 5f90b76..ca1363b 100644
--- a/libstdc++-v3/include/bits/sstream.tcc
+++ b/libstdc++-v3/include/bits/sstream.tcc
@@ -92,6 +92,9 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
#if _GLIBCXX_USE_CXX11_ABI
if ((this->epptr() - this->pbase()) < __capacity)
{
+#if _GLIBCXX_SANITIZER_ANNOTATE_STRING
+ _M_string.__annotate_grow(_M_string.size(), __capacity);
+#endif
// There is additional capacity in _M_string that can be used.
char_type* __base = const_cast<char_type*>(_M_string.data());
_M_pbump(__base, __base + __capacity, this->pptr() - this->pbase());
@@ -269,6 +272,9 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
if (!__testin)
this->setg(__endg, __endg, __endg);
}
+#if _GLIBCXX_USE_CXX11_ABI && _GLIBCXX_SANITIZER_ANNOTATE_STRING
+ _M_string.__annotate_grow(_M_string.size(), _M_string.capacity());
+#endif
}
template <class _CharT, class _Traits, class _Alloc>
@@ -287,6 +293,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
// Inhibit implicit instantiations for required instantiations,
// which are defined via explicit instantiations elsewhere.
+#if !_GLIBCXX_SANITIZE_STRING
#if _GLIBCXX_EXTERN_TEMPLATE
extern template class basic_stringbuf<char>;
extern template class basic_istringstream<char>;
@@ -300,6 +307,7 @@ _GLIBCXX_BEGIN_NAMESPACE_VERSION
extern template class basic_stringstream<wchar_t>;
#endif
#endif
+#endif // !_GLIBCXX_SANITIZE_STRING
_GLIBCXX_END_NAMESPACE_VERSION
} // namespace std
--
2.7.4