Re: ASAN test failures make compare_tests useless

[Alexander Potapenko <glider at google dot com>]

Not sure I understand what the problem is. Responded inline.

On Mon, Aug 18, 2014 at 9:43 AM, Yury Gribov <y.gribov@samsung.com> wrote:
> On 08/18/2014 09:42 AM, Yury Gribov wrote:
>>
>> On 08/16/2014 04:37 AM, Manuel LÃpez-IbÃÃez wrote:
>>>
>>> On the compile farm, ASAN tests seem to fail a lot like:
>>>
>>> FAIL: c-c++-common/asan/global-overflow-1.c   -O0  output pattern
>>> test, is ==31166==ERROR: AddressSanitizer failed to allocate
>>> 0xdfff0001000 (15392894357504) bytes at address 2008fff7000 (errno:
>>> 12)
>>> ==31166==ReserveShadowMemoryRange failed while trying to map
>>> 0xdfff0001000 bytes. Perhaps you're using ulimit -v
>>> , should match READ of size 1 at 0x[0-9a-f]+ thread T0.*(
Sounds like the tests do not even start up properly. No mmap failures
should be reported.

>>> The problem is that those addresses and sizes are very random,
The output pattern that must be printed has these addresses masked out
(note "0x[0-9a-f]+" in your report).
No other lines with varying addresses should be printed.

>>> so when
>>> I compare the test results of a pristine trunk with a patched one, I
>>> get:
>>>
>>> New tests that FAIL:
>>>
>>> unix//-m64: c-c++-common/asan/global-overflow-1.c   -O0  output
>>> pattern test, is ==12875==ERROR: AddressSanitizer failed to allocate
>>> 0xdfff0001000 (15392894357504) bytes at address 2008fff7000 (errno:
>>> 12)
>>> unix//-m64: c-c++-common/asan/global-overflow-1.c   -O0  output
>>> pattern test, is ==18428==ERROR: AddressSanitizer failed to allocate
>>> 0xdfff0001000 (15392894357504) bytes at address 2008fff7000 (errno:
>>> 12)
>>> [... hundreds of ASAN tests that failed...]
>>>
>>> Old tests that failed, that have disappeared: (Eeek!)
>>>
>>> unix//-m64: c-c++-common/asan/global-overflow-1.c   -O0  output
>>> pattern test, is ==30142==ERROR: AddressSanitizer failed to allocate
>>> 0xdfff0001000 (15392894357504) bytes at address 2008fff7000 (errno:
>>> 12)
>>> unix//-m64: c-c++-common/asan/global-overflow-1.c   -O0  output
>>> pattern test, is ==31166==ERROR: AddressSanitizer failed to allocate
>>> 0xdfff0001000 (15392894357504) bytes at address 2008fff7000 (errno:
>>> 12)
>>> [... the same hundreds of tests that already failed before...]
>>>
>>> The above makes very difficult to identify failures caused by my patch.
>>>
>>> Can we remove the "==...." part of the error? This way compare_tests
>>> will ignore the failures.
Am I understanding correctly that "==" in the test stdout has some
special meaning for compare_tests (whatever they are, I'm not really
familiar with GCC testing infrastructure)?
If so, this is quite a questionable choice (e.g. Valgrind also
prefixes the report lines with "==12345=="), and I don't see the point
in removing PIDs/addresses to please this script.


>>> Alternatively, I could patch compare_tests to sed out that part before
>>> comparing. Would that be acceptable?
>>>
>>> Cheers,
>>>
>>> Manuel.
>>>
>>
>> Added Sanitizer folks. Frankly it'd be cool if dumping PIDs and
>> addresses could be turned off.
>>
>
> Ok, this time actually added them.
>



-- 
Alexander Potapenko
Software Engineer
Google Moscow