Re: [PATCH] Fix sporadic failure in g++.dg/tsan/aligned_vs_unaligned_race.C

[Dmitry Vyukov <dvyukov at google dot com>]

On Wed, Jan 21, 2015 at 11:34 AM, Jakub Jelinek <jakub@redhat.com> wrote:
> On Wed, Jan 21, 2015 at 12:23:34PM +0400, Dmitry Vyukov wrote:
>> Hi Mike,
>>
>> Yes, I can quantify the cost. Is it very high.
>>
>> Here is the patch that I used:
>>
>> --- rtl/tsan_rtl.cc (revision 226644)
>> +++ rtl/tsan_rtl.cc (working copy)
>> @@ -709,7 +709,11 @@
>>  ALWAYS_INLINE USED
>>  void MemoryAccess(ThreadState *thr, uptr pc, uptr addr,
>>      int kAccessSizeLog, bool kAccessIsWrite, bool kIsAtomic) {
>>    u64 *shadow_mem = (u64*)MemToShadow(addr);
>> +
>> +  atomic_fetch_add((atomic_uint64_t*)shadow_mem, 0, memory_order_acq_rel);
>
> And the cost of adding that atomic_fetch_add guarded by
> if (__builtin_expect (someCondition, 0)) ?
> If that doesn't slow down the non-deterministic default case too much,
> that would allow users to choose what they prefer - much faster unreliable
> and slower deterministic.  Then for the gcc testsuite we could opt for the
> latter.

It's more complex than this.

1. In reality the cost can be higher. We will need to construct a
spin-mutex in shadow. Thus there will be contention between threads
(both waiting for the mutex and cache-line contention).
Currently threads don't always write to shadow (just read), so the
cache line can actually be shared between several threads accessing
the same shadow.

2. We don't have a spare bit in shadow for the mutex.

3. Asynchronous shadow memory flush can turn arbitrary regions of
shadow memory into zeroes. The shadow is especially designed to
tolerate this. This does not play well with mutexes.

4. That won't make multi-threaded programs deterministic. First, there
is at least one another issue in tsan -- it has only 4 slots to
remember previous memory accesses, they are evicted in effectively
random manner. Then, multi-threaded programs are non-deterministic by
themselves.

So there are lots of technical problems, significant slowdown and no
value for end users. I don't want to solve it for tests. The invisible
barrier is the right solution to make tests deterministic.



>> On the standard tsan benchmark that does 8-byte writes:
>> before:
>> [       OK ] DISABLED_BENCH.Mop8Write (1161 ms)
>> after:
>> [       OK ] DISABLED_BENCH.Mop8Write (5085 ms)
>>
>> So that's 338% slowdown.
>
>         Jakub