This is the mail archive of the mailing list for the GCC project.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [Patch 0/X] [WIP][RFC][libsanitizer] Introduce HWASAN to GCC

On Wed, Sep 11, 2019 at 9:37 AM Matthew Malcomson
<> wrote:
> On 11/09/19 12:53, Martin Liška wrote:
> > On 9/9/19 5:54 PM, Matthew Malcomson wrote:
> >> On 09/09/19 11:47, Martin Liška wrote:
> >>> On 9/6/19 4:46 PM, Matthew Malcomson wrote:
> >>>> Hello,
> >>>>
> >> As I understand it, `hwasan-abi=interceptor` vs `platform` is about
> >> adding such MTE emulation for "application code" or "platform code (e.g.
> >> kernel)" respectively.
> >
> > Hm, are you sure? Clang also uses -fsanitize=kernel-hwaddress which should
> > be equivalent to kernel-address for -fsanitize=address.
> >
> I'm not at all sure it's to do with the kernel ;-}
> Here's the commit that adds the flag.
>  From the commit message it seems the point is to distinguish between
> running on runtimes that natively support HWASAN (named the "platform"
> abi) and those where functions like malloc and pthread_create have to be
> intercepted (named the "interceptor" abi).
> I had assumed that targeting the kernel would be in the "platform"
> group, but it could easily not be the case.
> Considering the message form the below commit it seems that this is more
> targeted at instrumenting things like libc

With hwasan we tried a different approach from asan: instead of
intercepting libc we build it with sanitizer instrumentation, and rely
on a few hooks to update internal state of the tool on interesting
events, such as process startup, thread creation and destruction,
stack unwind (longjmp, vfork). This effectively puts hwasan _below_
libc (as in libc depends on libhwasan).

It has worked amazingly well for Android, where we aim to sanitize
most of platform code at once. Ex. ASan has this requirement that the
main executable needs to be built with ASan before any of the
libraries could - otherwise the tool will not be able to interpose
malloc/free symbols. As a consequence, when there are binaries that
can not be sanitized for any reason, we need to keep unsanitized
copies of all their transitive dependencies, and that turns into a
huge build/deployment mess. Hwasan approach avoids this problem by
making sure that the allocator is always there (because everything
depends on libc).

The downside, of course, is that this can not be used to sanitize a
single binary without a specially built libc. Hence the "interceptor"
ABI, which was an attempt to support running hwasan-instrumented
applications on regular, non-hwasan devices. We are not developing
this mode any longer, but it is used to run compiler-rt tests on

> I'm currently working on writing down the questions I plan to ask the
> developers of HWASAN in LLVM, I'll put this on the list :-)
> >>
> >>>
> >> There's an even more fundamental problem of accesses within the
> >> instrumented binary -- I haven't yet figured out how to remove the tag
> >> before accesses on architectures without the AArch64 TBI feature.
> >
> > Which should platforms like x86_64, right?
> Yes.
> As yet I haven't gotten anything working for architectures without TBI
> (everything except AArch64).
> This particular problem was one I was hoping for suggestions around (my
> first of the questions in my cover letter).

We have support for hwasan on x86_64 in LLVM (by removing tags before
accesses), but it is not really practical because any library built
without instrumentation is a big source of false positives. Even, say,
libc++/libstdc++. We use it exclusively for tests.

> >>>>
> >>>> The current patch series is far from complete, but I'm posting the current state
> >>>> to provide something to discuss at the Cauldron next week.
> >>>>
> >>>> In its current state, this sanitizer only works on AArch64 with a custom kernel
> >>>> to allow tagged pointers in system calls.  This is discussed in the below link
> >>>> -- the custom kernel allows
> >>>> tagged pointers in syscalls.
> >>>
> >>> Can you be please more specific. Is the MTE in upstream linux kernel? If so,
> >>> starting from which version?
> >>
> >> I find I can only make complicated statements remotely clear in bullet
> >> points ;-)
> >>
> >> What I was trying to say was:
> >> - HWASAN from this patch series requires AArch64 TBI.
> >>     (I have not handled architectures without TBI)
> >> - The upstream kernel does not accept tagged pointers in syscalls.
> >>     (programs that use TBI must currently clear tags before passing
> >>      pointers to the kernel)
> >
> > I know that in case of ASAN, the libasan provides wrappers (interceptors) for various glibc
> > functions that are often system calls. Similar wrappers are probably used in HWASAN
> > and so that one can create the memory pointer tags.
> >
> >> - This patch series doesn't include any way to avoid passing tagged
> >>     pointers to syscalls.
> >
> > I bet LLVM has the same problem so I would expect a handling in the interceptors.
> >
> I'm pretty sure this problem hasn't been solved with interceptors.
> The android page describing hwasan specifically mentions the requirement
> of a Linux kernel accepting tagged pointers, and I believe this is the
> most supported environment.
> "HWASan requires the Linux kernel to accept tagged pointers in system
> call arguments."
> Also, there are surprisingly few interceptors defined in libhwasan.
> Thanks,
> Matthew
> >> - Hence on order to test the sanitizer I'm using a kernel that has been
> >>     patched to accept tagged pointers in many syscalls.
> >> - The link to the site is just another source describing the
> >>     same requirement.
> >>
> >>
> >> The support for the relaxed ABI (of accepting tagged pointers in various
> >> syscalls in the kernel) is being discussed on the kernel mailing list,
> >> the latest patchset I know of is here:
> >>

The main patchset is this one:

AFAIK, it's expected to go in the next merge window.

> >
> > Thanks for pointer.
> >

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]