Re: [google] AddressSanitizer for gcc, first attempt. (issue 5272048)

[Xinliang David Li <davidxl at google dot com>]

On Tue, Nov 1, 2011 at 12:41 PM, Diego Novillo <dnovillo@google.com> wrote:
> On 11-11-01 15:34 , Xinliang David Li wrote:
>
>>> Right before pass_expand? ?In init_optimization_passes(), look for
>>> NEXT_PASS
>>> (pass_expand). ?That's RTL generation. ?Somewhere before that.
>>>
>>
>> Why?
>
> The idea was to experiment where to best place ASAN to avoid instrumenting
> too much. ?If we schedule it really late, then we may save ourselves some
> unnecessary instrumentation.
>

It needs to be balanced -- on one hand it needs to be as late as
possible so that as few memory references (dynamically executed) as
possible are instrumented. On the other hand, early enough so that the
instrumented code can be optimized sufficiently.

> Though, I still think ASAN should never open code the library calls
> directly. ?Rather, it should emit straight-code gimple that can be better
> understood and optimized away.

that depends on the library function themselves -- if they are
trivial, inline sequence should be generated.

>
>
>>> TARGET_MEM_REFs are converted to RTL mems during RTL expansion.
>>>
>>
>> What? they will still be seen by asan which can not be handled (e.g,
>> creating address expression out of it).
>
> So, it needs to run before TMRs are introduced then. ?*shrug*.
>

yes it should be before ivopt as discussed.

David