Bug 64435 - [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel <= 3.15
Summary: [5 Regression] Bootstrap failure in libsanitizer on AArch64 with Linux kernel...
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: sanitizer (show other bugs)
Version: 5.0
: P1 major
Target Milestone: 5.0
Assignee: Not yet assigned to anyone
URL:
Keywords: build
Depends on: 64131
Blocks:
  Show dependency treegraph
 
Reported: 2014-12-29 15:41 UTC by David Abdurachmanov
Modified: 2015-01-26 15:18 UTC (History)
5 users (show)

See Also:
Host:
Target: aarch64
Build:
Known to work:
Known to fail:
Last reconfirmed: 2015-01-18 00:00:00


Attachments
RFC patch (tested) (620 bytes, patch)
2014-12-29 18:21 UTC, David Abdurachmanov
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description David Abdurachmanov 2014-12-29 15:41:20 UTC
I finally started testing GCC trunk (5.0.0) on AArch64 before release is made.

I quickly find out that I couldn't bootstrap GCC on APM Mustang board with APM Linux images (F19 + 3.12 kernel), but it worked fine with QEMU + F21 + 3.17 kernel). Failure details are below.

The culprit is 34c65c43f1518bf85f93526ad373adc6a683b4c5 from Linux repository. Commit details are provided below.

Simply put __sanitizer___kernel_{uid,gid}_t depends on kernel version. <=3.15 it's unsigned int and >=3.16 it's unsigned short.

##### GCC trunk failure #####

In file included from ../../../../libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc:176:0:
../../../../libsanitizer/sanitizer_common/sanitizer_internal_defs.h:272:72: error: size of array 'assertion_failed__1006' is negative
     typedef char IMPL_PASTE(assertion_failed_##_, line)[2*(int)(pred)-1]
                                                                        ^
../../../../libsanitizer/sanitizer_common/sanitizer_internal_defs.h:266:30: note: in expansion of macro 'IMPL_COMPILER_ASSERT'
 #define COMPILER_CHECK(pred) IMPL_COMPILER_ASSERT(pred, __LINE__)
                              ^
../../../../libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.h:1285:3: note: in expansion of macro 'COMPILER_CHECK'
   COMPILER_CHECK(sizeof(__sanitizer_##TYPE) == sizeof(TYPE))
   ^
../../../../libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc:1006:1: note: in expansion of macro 'CHECK_TYPE_SIZE'
 CHECK_TYPE_SIZE(__kernel_old_uid_t);
 ^
../../../../libsanitizer/sanitizer_common/sanitizer_internal_defs.h:272:72: error: size of array 'assertion_failed__1007' is negative
     typedef char IMPL_PASTE(assertion_failed_##_, line)[2*(int)(pred)-1]
                                                                        ^
../../../../libsanitizer/sanitizer_common/sanitizer_internal_defs.h:266:30: note: in expansion of macro 'IMPL_COMPILER_ASSERT'
 #define COMPILER_CHECK(pred) IMPL_COMPILER_ASSERT(pred, __LINE__)
                              ^
../../../../libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.h:1285:3: note: in expansion of macro 'COMPILER_CHECK'
   COMPILER_CHECK(sizeof(__sanitizer_##TYPE) == sizeof(TYPE))
   ^
../../../../libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc:1007:1: note: in expansion of macro 'CHECK_TYPE_SIZE'
 CHECK_TYPE_SIZE(__kernel_old_gid_t);
 ^
make[4]: *** [sanitizer_platform_limits_posix.lo] Error 1
make[4]: *** Waiting for unfinished jobs....

##### Linux Commit #####

commit 34c65c43f1518bf85f93526ad373adc6a683b4c5
Author: Will Deacon <will.deacon@arm.com>
Date:   Mon Jun 2 11:47:29 2014 +0100

    arm64: uid16: fix __kernel_old_{gid,uid}_t definitions

    Whilst native arm64 applications don't have the 16-bit UID/GID syscalls
    wired up, compat tasks can still access them. The 16-bit wrappers for
    these syscalls use __kernel_old_uid_t and __kernel_old_gid_t, which must
    be 16-bit data types to maintain compatibility with the 16-bit UIDs used
    by compat applications.

    This patch defines 16-bit __kernel_old_{gid,uid}_t types for arm64
    instead of using the 32-bit types provided by asm-generic.

    Signed-off-by: Will Deacon <will.deacon@arm.com>
    Acked-by: Arnd Bergmann <arnd@arndb.de>
    Cc: <stable@vger.kernel.org>
    Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>

diff --git a/arch/arm64/include/asm/Kbuild b/arch/arm64/include/asm/Kbuild
index 42c7eec..0b3fcf8 100644
--- a/arch/arm64/include/asm/Kbuild
+++ b/arch/arm64/include/asm/Kbuild
@@ -30,7 +30,6 @@ generic-y += msgbuf.h
 generic-y += mutex.h
 generic-y += pci.h
 generic-y += poll.h
-generic-y += posix_types.h
 generic-y += preempt.h
 generic-y += resource.h
 generic-y += rwsem.h
diff --git a/arch/arm64/include/uapi/asm/posix_types.h b/arch/arm64/include/uapi/asm/posix_types.h
new file mode 100644
index 0000000..7985ff6
--- /dev/null
+++ b/arch/arm64/include/uapi/asm/posix_types.h
@@ -0,0 +1,10 @@
+#ifndef __ASM_POSIX_TYPES_H
+#define __ASM_POSIX_TYPES_H
+
+typedef unsigned short __kernel_old_uid_t;
+typedef unsigned short __kernel_old_gid_t;
+#define __kernel_old_uid_t __kernel_old_uid_t
+
+#include <asm-generic/posix_types.h>
+
+#endif /*  __ASM_POSIX_TYPES_H */
Comment 1 David Abdurachmanov 2014-12-29 18:21:49 UTC
Created attachment 34356 [details]
RFC patch (tested)

Bootstrapped on aarch64-linux-gnu machine with F19 + 3.12 and on QEMU with F21 + 3.17.
Comment 2 David Abdurachmanov 2014-12-29 18:25:01 UTC
linux/version.h does not bring any additional kernel headers, its fully standalone and seems fine to include.

There might be a problem is someone builds a distribution with GCC 5 and kernel <=3.15 and then decides to update the kernel creating mismatch with libsanitizer.
Comment 3 David Abdurachmanov 2015-01-09 11:54:46 UTC
The patch has been submitted to LLVM/compiler-rt and approved. Not yet committed, pending testing with Clang.

http://reviews.llvm.org/D6854
Comment 4 Jakub Jelinek 2015-01-13 15:44:56 UTC
So, no progress upstream yet?
Comment 5 Christophe Lyon 2015-01-13 21:17:42 UTC
Not yet. My AArch64 board has just been brought back online, and I managed to execute the asan tests in clang/llvm. I can see failure, I'll look at them in more details.
Comment 6 Jakub Jelinek 2015-01-18 12:38:19 UTC
Well, for the old uid/git we can temporarily also just cherry pick upstream r223925.  Even with that patch I'm running into:
../../../../libsanitizer/sanitizer_common/sanitizer_internal_defs.h:272:72: error: size of array 'assertion_failed__1062' is negative
     typedef char IMPL_PASTE(assertion_failed_##_, line)[2*(int)(pred)-1]
                                                                        ^
../../../../libsanitizer/sanitizer_common/sanitizer_internal_defs.h:266:30: note: in expansion of macro 'IMPL_COMPILER_ASSERT'
 #define COMPILER_CHECK(pred) IMPL_COMPILER_ASSERT(pred, __LINE__)
                              ^
../../../../libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.h:1288:3: note: in expansion of macro 'COMPILER_CHECK'
   COMPILER_CHECK(sizeof(((__sanitizer_##CLASS *) NULL)->MEMBER) == \
   ^
../../../../libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc:1062:1: note: in expansion of macro 'CHECK_SIZE_AND_OFFSET'
 CHECK_SIZE_AND_OFFSET(ipc_perm, mode);
Comment 7 Jakub Jelinek 2015-01-18 13:50:50 UTC
And the reason for that is that aarch64 again changed ABI, what a stable port :(.

https://sourceware.org/git/?p=glibc.git;a=commit;h=5c40c3bab2fddaca8cfe12d75944d1fef8adf1a4
Comment 8 David Abdurachmanov 2015-01-18 19:07:27 UTC
I will finish testing my patch for upstream next week. I was busy with other tasks.

AArch64 is young, this kind of things are bound to happen :/
Comment 9 Jakub Jelinek 2015-01-18 19:11:35 UTC
I've used:
--- libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.h	2014-11-14 00:10:33.735060963 +0100
+++ libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.h	2015-01-18 14:43:05.812763769 +0100
@@ -167,7 +167,7 @@ namespace __sanitizer {
     unsigned __seq;
     u64 __unused1;
     u64 __unused2;
-#elif defined(__mips__)
+#elif defined(__mips__) || defined(__aarch64__)
     unsigned int mode;
     unsigned short __seq;
     unsigned short __pad1;
--- libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc	2014-11-14 00:10:33.735060963 +0100
+++ libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc	2015-01-18 14:47:18.901482608 +0100
@@ -1055,7 +1055,10 @@ CHECK_SIZE_AND_OFFSET(ipc_perm, uid);
 CHECK_SIZE_AND_OFFSET(ipc_perm, gid);
 CHECK_SIZE_AND_OFFSET(ipc_perm, cuid);
 CHECK_SIZE_AND_OFFSET(ipc_perm, cgid);
+#if !defined(__aarch64__) || !SANITIZER_LINUX || __GNUC_PREREQ (2, 21)
+/* On aarch64 glibc 2.20 and earlier provided incorrect mode field.  */
 CHECK_SIZE_AND_OFFSET(ipc_perm, mode);
+#endif
 
 CHECK_TYPE_SIZE(shmid_ds);
 CHECK_SIZE_AND_OFFSET(shmid_ds, shm_perm);

to fix the #c6 issue.  That said, seems asan is totally broken at least in Fedora 22/aarch64, but at least it builds with this
and r223925 cherry-pick.
Pretty much every test dies with
AddressSanitizer CHECK failed: ../../../../libsanitizer/asan/asan_poisoning.cc:24 "((AddrIsInMem(addr))) != (0)
Comment 10 Andrew Pinski 2015-01-18 19:37:06 UTC
(In reply to Jakub Jelinek from comment #7)
> And the reason for that is that aarch64 again changed ABI, what a stable
> port :(.
> 
> https://sourceware.org/git/?p=glibc.git;a=commit;
> h=5c40c3bab2fddaca8cfe12d75944d1fef8adf1a4

This was recorded as bug 64131.  I filed it when I updated my glibc sources and found Asan was broken.

Note ILP32 is also broken with Asan.
Comment 11 Christophe Lyon 2015-01-18 19:51:47 UTC
(In reply to David Abdurachmanov from comment #8)
> I will finish testing my patch for upstream next week. I was busy with other
> tasks.
> 
How are you going to test it?

FYI, I am now able to run the LLVM/Asan tests on a AArch64 system, and I have noticed numerous failures. I'm currently debugging the Asan instrumentation code which is generating memory accesses to invalid memory areas.
Comment 12 Jakub Jelinek 2015-01-19 08:39:59 UTC
Author: jakub
Date: Mon Jan 19 08:39:27 2015
New Revision: 219833

URL: https://gcc.gnu.org/viewcvs?rev=219833&root=gcc&view=rev
Log:
	PR sanitizer/64435
	* sanitizer_common/sanitizer_platform_limits_posix.cc: Cherry pick
	upstream r223925.

Modified:
    trunk/libsanitizer/ChangeLog
    trunk/libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc
Comment 13 Jakub Jelinek 2015-01-19 16:53:02 UTC
In the #c9 patch obviously I meant to write __GLIBC_PREREQ (2, 21) instead of
__GNUC_PREREQ (2, 21), finger memory apparently doesn't work always well.
And there probably needs to be
#ifndef __GLIBC_PREREQ
#define __GLIBC_PREREQ(major, minor) 0
#endif
somewhere before that, as otherwise it will fail on non-Linux.
Comment 14 Jakub Jelinek 2015-01-19 17:29:03 UTC
And the reason why at least in RHEL and Fedora asan doesn't work at all is:
cat /proc/self/maps; ASAN_OPTIONS=verbosity=1 ./null-deref-1.exe
00400000-00410000 r-xp 00000000 fd:00 134631897                          /usr/bin/cat
00410000-00420000 r--p 00000000 fd:00 134631897                          /usr/bin/cat
00420000-00430000 rw-p 00010000 fd:00 134631897                          /usr/bin/cat
142c0000-142f0000 rw-p 00000000 00:00 0                                  [heap]
3ff929d0000-3ff99270000 r--p 00000000 fd:00 4086                         /usr/lib/locale/locale-archive
3ff99270000-3ff993d0000 r-xp 00000000 fd:00 203072141                    /usr/lib64/libc-2.17.so
3ff993d0000-3ff993e0000 r--p 00150000 fd:00 203072141                    /usr/lib64/libc-2.17.so
3ff993e0000-3ff993f0000 rw-p 00160000 fd:00 203072141                    /usr/lib64/libc-2.17.so
3ff99410000-3ff99420000 r--p 00000000 00:00 0                            [vvar]
3ff99420000-3ff99430000 r-xp 00000000 00:00 0                            [vdso]
3ff99430000-3ff99450000 r-xp 00000000 fd:00 201341988                    /usr/lib64/ld-2.17.so
3ff99450000-3ff99460000 r--p 00010000 fd:00 201341988                    /usr/lib64/ld-2.17.so
3ff99460000-3ff99470000 rw-p 00020000 fd:00 201341988                    /usr/lib64/ld-2.17.so
3fff0680000-3fff06b0000 rw-p 00000000 00:00 0                            [stack]
==2862==Parsed ASAN_OPTIONS: verbosity=1
==2862==AddressSanitizer: libc interceptors initialized
|| `[0x002000000000, 0x007fffffffff]` || HighMem    ||
|| `[0x001400000000, 0x001fffffffff]` || HighShadow ||
|| `[0x001200000000, 0x0013ffffffff]` || ShadowGap  ||
|| `[0x001000000000, 0x0011ffffffff]` || LowShadow  ||
|| `[0x000000000000, 0x000fffffffff]` || LowMem     ||
MemToShadow(shadow): 0x001200000000 0x00123fffffff 0x001280000000 0x0013ffffffff
redzone=16
max_redzone=2048
quarantine_size=256M
malloc_context_size=30
SHADOW_SCALE: 3
SHADOW_GRANULARITY: 8
SHADOW_OFFSET: 1000000000
==2862==Installed the sigaction for signal 11
==2862==AddressSanitizer CHECK failed: ../../../../libsanitizer/asan/asan_poisoning.cc:24 "((AddrIsInMem(addr))) != (0)" (0x0, 0x0)

https://github.com/torvalds/linux/blob/master/arch/arm64/include/asm/memory.h
https://github.com/torvalds/linux/blob/master/arch/arm64/Kconfig
reveals that aarch64 can be on Linux configured to support 39, 42 or 48 bits virtual address space.  The current libsanitizer/asan/ and gcc/config/aarch64/aarch64.c (aarch64_asan_shadow_offset) seems to be okay only for the 39 bits virtual address space, while Fedora/RHEL apparently use 42 bits VA.  Wonder if aarch64 couldn't use a layout closer to what x86_64 uses for asan, with shadow offset low 0x7fff8000, which is flexible to different sizes of the virtual address space.
Comment 15 Andrew Pinski 2015-01-19 17:55:03 UTC
(In reply to Jakub Jelinek from comment #14)

> https://github.com/torvalds/linux/blob/master/arch/arm64/include/asm/memory.h
> https://github.com/torvalds/linux/blob/master/arch/arm64/Kconfig
> reveals that aarch64 can be on Linux configured to support 39, 42 or 48 bits
> virtual address space.  The current libsanitizer/asan/ and
> gcc/config/aarch64/aarch64.c (aarch64_asan_shadow_offset) seems to be okay
> only for the 39 bits virtual address space, while Fedora/RHEL apparently use
> 42 bits VA.  Wonder if aarch64 couldn't use a layout closer to what x86_64
> uses for asan, with shadow offset low 0x7fff8000, which is flexible to
> different sizes of the virtual address space.

Right selecting 64k pages causes this.  Asan is known to be broken with the different virtual address space due page size differences.  PowerPC has a similar issue IIRC.  x86_64 does not have this issue as there is only one page size and not different virtual address spaces there.

I think ASAN for AARCH64 needs testing on multiple virtual address sizes before it can be turned on by default.  Note most distros uses 64k pages (Fedora for an example) but some don't MontaVista CG7 for X-gene does not due to needing to allow ARMv7 programs to run.
Comment 16 Jakub Jelinek 2015-01-19 17:56:24 UTC
Trying a PIE on aarch64 reveals:
2aaca300000-2aaca310000 r-xp 00000000 fd:00 17567415                     /tmp/m
2aaca310000-2aaca320000 rw-p 00000000 fd:00 17567415                     /tmp/m
mappings alongside of 3ffXXXXXXXX, thus we need low mem from 0 to 2GB or 4GB?,
then maybe some middle memory, and then high memory.
Comment 17 Jakub Jelinek 2015-01-19 18:01:23 UTC
PPC64 actually supports both 44 and 46 bit address space:

uptr GetMaxVirtualAddress() {
#if SANITIZER_WORDSIZE == 64
# if defined(__powerpc64__)
  // On PowerPC64 we have two different address space layouts: 44- and 46-bit.
  // We somehow need to figure out which one we are using now and choose
  // one of 0x00000fffffffffffUL and 0x00003fffffffffffUL.
  // Note that with 'ulimit -s unlimited' the stack is moved away from the top
  // of the address space, so simply checking the stack address is not enough.
  // This should (does) work for both PowerPC64 Endian modes.
  return (1ULL << (MostSignificantSetBitIndex(GET_CURRENT_FRAME()) + 1)) - 1;
# elif defined(__aarch64__)
  return (1ULL << 39) - 1;
# elif defined(__mips64)
  return (1ULL << 40) - 1;
# else
  return (1ULL << 47) - 1;  // 0x00007fffffffffffUL;
# endif
#else  // SANITIZER_WORDSIZE == 32
  uptr res = (1ULL << 32) - 1;  // 0xffffffff;
  if (!common_flags()->full_address_space)
    res -= GetKernelAreaSize();
  CHECK_LT(reinterpret_cast<uptr>(&res), res);
  return res;
#endif  // SANITIZER_WORDSIZE
}

it is just that aarch64 hardcodes this right now.  Suppose doing something like ppc64 does could work.
Comment 18 Andrew Pinski 2015-01-19 18:09:38 UTC
(In reply to Jakub Jelinek from comment #17)
> PPC64 actually supports both 44 and 46 bit address space:
> 
> uptr GetMaxVirtualAddress() {
> #if SANITIZER_WORDSIZE == 64
> # if defined(__powerpc64__)
>   // On PowerPC64 we have two different address space layouts: 44- and
> 46-bit.
>   // We somehow need to figure out which one we are using now and choose
>   // one of 0x00000fffffffffffUL and 0x00003fffffffffffUL.
>   // Note that with 'ulimit -s unlimited' the stack is moved away from the
> top
>   // of the address space, so simply checking the stack address is not
> enough.
>   // This should (does) work for both PowerPC64 Endian modes.
>   return (1ULL << (MostSignificantSetBitIndex(GET_CURRENT_FRAME()) + 1)) - 1;
> # elif defined(__aarch64__)
>   return (1ULL << 39) - 1;
> # elif defined(__mips64)
>   return (1ULL << 40) - 1;
...
> it is just that aarch64 hardcodes this right now.  Suppose doing something
> like ppc64 does could work.

Even though this is not listed here and does not really belong in this bug but MIPS64 should be done the same way since it has many different virtual address sizes as it has many different page sizes too (4k, 8k, 16k, 32k, and 64k).  I wonder why ASAN folks don't just always use the way PowerPC is done.
Comment 19 Jakub Jelinek 2015-01-19 18:33:36 UTC
So, with:
--- libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.h.jj	2014-11-14 00:10:33.000000000 +0100
+++ libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.h	2015-01-19 19:22:43.777780408 +0100
@@ -167,7 +167,7 @@ namespace __sanitizer {
     unsigned __seq;
     u64 __unused1;
     u64 __unused2;
-#elif defined(__mips__)
+#elif defined(__mips__) || defined(__aarch64__)
     unsigned int mode;
     unsigned short __seq;
     unsigned short __pad1;
--- libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc.jj	2015-01-19 09:39:09.000000000 +0100
+++ libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc	2015-01-19 19:21:57.608564325 +0100
@@ -1059,7 +1059,13 @@ CHECK_SIZE_AND_OFFSET(ipc_perm, uid);
 CHECK_SIZE_AND_OFFSET(ipc_perm, gid);
 CHECK_SIZE_AND_OFFSET(ipc_perm, cuid);
 CHECK_SIZE_AND_OFFSET(ipc_perm, cgid);
+#ifndef __GLIBC_PREREQ
+#define __GLIBC_PREREQ(x, y) 0
+#endif
+#if !defined(__aarch64__) || !SANITIZER_LINUX || __GLIBC_PREREQ (2, 21)
+/* On aarch64 glibc 2.20 and earlier provided incorrect mode field.  */
 CHECK_SIZE_AND_OFFSET(ipc_perm, mode);
+#endif
 
 CHECK_TYPE_SIZE(shmid_ds);
 CHECK_SIZE_AND_OFFSET(shmid_ds, shm_perm);
--- libsanitizer/sanitizer_common/sanitizer_posix.cc.jj	2014-11-14 00:10:33.000000000 +0100
+++ libsanitizer/sanitizer_common/sanitizer_posix.cc	2015-01-19 19:24:14.636237703 +0100
@@ -76,16 +76,15 @@ static uptr GetKernelAreaSize() {
 
 uptr GetMaxVirtualAddress() {
 #if SANITIZER_WORDSIZE == 64
-# if defined(__powerpc64__)
+# if defined(__powerpc64__) || defined(__aarch64__)
   // On PowerPC64 we have two different address space layouts: 44- and 46-bit.
   // We somehow need to figure out which one we are using now and choose
   // one of 0x00000fffffffffffUL and 0x00003fffffffffffUL.
   // Note that with 'ulimit -s unlimited' the stack is moved away from the top
   // of the address space, so simply checking the stack address is not enough.
   // This should (does) work for both PowerPC64 Endian modes.
+  // Similarly, aarch64 has multiple address space layouts: 39, 42 and 48-bit.
   return (1ULL << (MostSignificantSetBitIndex(GET_CURRENT_FRAME()) + 1)) - 1;
-# elif defined(__aarch64__)
-  return (1ULL << 39) - 1;
 # elif defined(__mips64)
   return (1ULL << 40) - 1;
 # else

the layout looks reasonable:
|| `[0x009000000000, 0x03ffffffffff]` || HighMem    ||
|| `[0x002200000000, 0x008fffffffff]` || HighShadow ||
|| `[0x001200000000, 0x0021ffffffff]` || ShadowGap  ||
|| `[0x001000000000, 0x0011ffffffff]` || LowShadow  ||
|| `[0x000000000000, 0x000fffffffff]` || LowMem     ||
MemToShadow(shadow): 0x001200000000 0x00123fffffff 0x001440000000 0x0021ffffffff
and should accomodate the different addresses I saw for 42-bit address space so far: <4GB, 0x10000000000 (place where libraries are mapped with ulimit -s unlimited), 0x2aaXXXXXXXX (PIEs), 0x3ffXXXXXXXX (stack, normal mmap area for limited ulimit -s).
But it still doesn't work:
==4746==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000028 (pc 0x0000004008f8 bp 0x03fff8078fc0 sp 0x03fff8078fc0 T0)
==4746==AddressSanitizer CHECK failed: ../../../../libsanitizer/sanitizer_common/sanitizer_allocator.h:835 "((res)) < ((kNumPossibleRegions))" (0x3ffa33, 0x80000)

I've tried to change
#if SANITIZER_CAN_USE_ALLOCATOR64
# if defined(__powerpc64__)
const uptr kAllocatorSpace =  0xa0000000000ULL;
const uptr kAllocatorSize  =  0x20000000000ULL;  // 2T.
# else
const uptr kAllocatorSpace = 0x600000000000ULL;
const uptr kAllocatorSize  =  0x40000000000ULL;  // 4T.
# endif
from the non-ppc64 values to the ppc64 values (i.e. added " || defined(__aarch64__)"), but strangely that didn't change anything.  So I'm out of ideas on what else needs to be tweaked.  Kostya?
Comment 20 Christophe Lyon 2015-01-19 20:13:17 UTC
(In reply to Jakub Jelinek from comment #14)
> 
> https://github.com/torvalds/linux/blob/master/arch/arm64/include/asm/memory.h
> https://github.com/torvalds/linux/blob/master/arch/arm64/Kconfig
> reveals that aarch64 can be on Linux configured to support 39, 42 or 48 bits
> virtual address space.  The current libsanitizer/asan/ and
> gcc/config/aarch64/aarch64.c (aarch64_asan_shadow_offset) seems to be okay
> only for the 39 bits virtual address space, while Fedora/RHEL apparently use
> 42 bits VA.  Wonder if aarch64 couldn't use a layout closer to what x86_64
> uses for asan, with shadow offset low 0x7fff8000, which is flexible to
> different sizes of the virtual address space.

Thanks for pointing that, I wasn't aware of it when I worked on the initial port. My platform had 39 bits virtual address space, and I didn't notice there were other possibilities.
Comment 21 Jakub Jelinek 2015-01-19 21:49:01 UTC
Ugh, there is also this junk:
// By default we allow to use SizeClassAllocator64 on 64-bit platform.
// But in some cases (e.g. AArch64's 39-bit address space) SizeClassAllocator64
// does not work well and we need to fallback to SizeClassAllocator32.
// For such platforms build this code with -DSANITIZER_CAN_USE_ALLOCATOR64=0 or
// change the definition of SANITIZER_CAN_USE_ALLOCATOR64 here.
#ifndef SANITIZER_CAN_USE_ALLOCATOR64
# if defined(__aarch64__) || defined(__mips64)
#  define SANITIZER_CAN_USE_ALLOCATOR64 0
# else
#  define SANITIZER_CAN_USE_ALLOCATOR64 (SANITIZER_WORDSIZE == 64)
# endif
#endif

It would be nice to know why that has been added.
Was that just because
#if SANITIZER_CAN_USE_ALLOCATOR64
# if defined(__powerpc64__)
const uptr kAllocatorSpace =  0xa0000000000ULL;
const uptr kAllocatorSize  =  0x20000000000ULL;  // 2T.
# else
const uptr kAllocatorSpace = 0x600000000000ULL;
const uptr kAllocatorSize  =  0x40000000000ULL;  // 4T.
# endif
has not been adjusted for the port?
What is the minimum kAllocatorSize that would work?  Given the very small 39-bit VA option on aarch64, I think if the allocator64 memory has to be normal memory with shadow, we could only use say something like
kAllocatorSpace 0x800000000 and kAllocatorSize the same value, i.e. 32GB.  Would that be enough?

There is also the:
// The range of addresses which can be returned my mmap.
// FIXME: this value should be different on different platforms,
// e.g. on AArch64 it is most likely (1ULL << 39). Larger values will still work
// but will consume more memory for TwoLevelByteMap.
#if defined(__aarch64__)
# define SANITIZER_MMAP_RANGE_SIZE FIRST_32_SECOND_64(1ULL << 32, 1ULL << 39)
#else
# define SANITIZER_MMAP_RANGE_SIZE FIRST_32_SECOND_64(1ULL << 32, 1ULL << 47)
#endif
which is again wrong for aarch64.  Does it really has to be compile time constant?  The 1ULL << 47 is also wrong ppc64, mips64, isn't it?
Comment 22 Jakub Jelinek 2015-01-20 08:10:24 UTC
Or perhaps we could change SizeClassAllocator64 so that on some architectures it could use variable bounds.  The fact that they are template parameters makes this harder, but perhaps we could say that if some template
parameters are zero then some variable is used instead.
Use kSpaceBeg ? kSpaceBeg : kSpaceBegVar instead of kSpaceBeg and
kSpaceSize ? kSpaceSize : kSpaceSizeVar instead of kSpace (perhaps put into some
method).  For architectures where there is no such big variability of address
space sizes it could stay constant, while for aarch64 we could decide during
asan initialization, after finding out how large address space we have.
So for say the 39-bit VA we could use 0x2000000000 to 0x3fffffffff (i.e. 128GB
for the allocator), which is not possible for 42-bit VA, because that is shadow
gap and high shadow.  And for 42-bit VA we could use say 0x10000000000 to
0x2ffffffffff (i.e. 2TB).
Comment 23 Kostya Serebryany 2015-01-20 16:53:02 UTC
Can we move the discussion to the asan issue tracker? 
https://code.google.com/p/address-sanitizer/issues/list

There is already quite a bit of aarch64 discussion at https://code.google.com/p/address-sanitizer/issues/detail?id=246

Not all of the asan developers are looking at the gcc buganizer and the fix will need to go to the upstream anyway.
Comment 24 Jakub Jelinek 2015-01-21 21:22:00 UTC
Author: jakub
Date: Wed Jan 21 21:21:27 2015
New Revision: 219968

URL: https://gcc.gnu.org/viewcvs?rev=219968&root=gcc&view=rev
Log:
	PR sanitizer/64435
	* sanitizer_common/sanitizer_platform_limits_posix.h: Cherry pick
	upstream r226637.
	* sanitizer_common/sanitizer_platform_limits_posix.cc: Likewise.
	* sanitizer_common/sanitizer_posix.cc: Cherry pick upstream r226639.

Modified:
    trunk/libsanitizer/ChangeLog
    trunk/libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.cc
    trunk/libsanitizer/sanitizer_common/sanitizer_platform_limits_posix.h
    trunk/libsanitizer/sanitizer_common/sanitizer_posix.cc
Comment 25 Jakub Jelinek 2015-01-26 15:18:04 UTC
The bootstrap failure should be fixed now.  ASAN doesn't work at all with != 39 bits of virtual address space though.