Re: [PATCH] Frame pointer for arm with THUMB2 mode

[Wilco Dijkstra <Wilco dot Dijkstra at arm dot com>]

Hi Denis,

> We are working on applying Address/LeakSanitizer for the full Tizen OS
> distribution. It's about ~1000 packages, ASan/LSan runtime is installed 
> to ld.so.preload. As we know ASan/LSan has interceptors for 
> allocators/deallocators such as (malloc/realloc/calloc/free) and so on.
> On every allocation from user space program, ASan calls
> GET_STACK_TRACE_MALLOC;
> which unwinds the stack frame, and by default uses frame based stack
> unwinder. So, it requires to build with "-fno-omit-frame-pointer", 
> switching it to default unwinder really hits the performance in our case.

So this sounds like the first thing to do is reducing the size of the stack traces.
The default is 30 which is far larger than useful. Using 1 for example should
always be fast (since you can use __builtin_return_address(0)) and still get
the function that called malloc. Also if the unwinder happens to be too slow,
it should be optimized and caching added etc.

>> Doing real unwinding is also far more accurate than frame pointer based
> > unwinding (the latter doesn't handle leaf functions correctly, 
> entry/exit in
> > non-leaf functions and shrinkwrapped functions - and this breaks 
> callgraph
> > profiling).
>
> I agree, but in our case, all interceptors for allocators are
> leaf functions, so the frame based stack unwinder works well for us.

Yes a frame chain would work for this case. But it's not currently supported.

> By default we build packages with ("-marm" "-fno-omit-frame-pointer"),
> because need frame based stack unwinder for every allocation, as I said
> before. As we know GCC sets fp to lr on the stack with 
> ("-fno-omit-frame-pointer" and "-marm") and I don't really know why.
> But the binary size is bigger than for thumb, so, we cannot use default 
> thumb frame pointer and want to reduce binary size for the full 
> sanitized image.

The issue is that the frame pointer and frame chain always add a large
overhead even when you do not use any sanitizers. This is especially bad
for the proposed patch - you lose much of the benefit of using Thumb-2...

Using normal unwinding means your code runs at full speed and still can be
used by the sanitizer.

> In other case clang works the same way, as I offered at the patch.
> It has the same issue, but it was fixed at the end of 2017
> https://bugs.llvm.org/show_bug.cgi?id=18505 (The topics starts from
> discussion about APCS, but it is not the main point.)
>
> Also, unresolved issue related to this
> https://github.com/google/sanitizers/issues/640

Adding support for a frame chain would require an ABI change. It would have to 
work across GCC, LLVM, Arm, Thumb-1 and Thumb-2 - not a trivial amount of
effort.

Wilco