This is the mail archive of the
gcc-patches@gcc.gnu.org
mailing list for the GCC project.
Re: [PATCH v2] aarch64: Add split-stack initial support
- From: Adhemerval Zanella <adhemerval dot zanella at linaro dot org>
- To: Wilco Dijkstra <Wilco dot Dijkstra at arm dot com>, Jiong Wang <Jiong dot Wang at arm dot com>
- Cc: nd <nd at arm dot com>, GCC Patches <gcc-patches at gcc dot gnu dot org>
- Date: Mon, 7 Nov 2016 16:59:21 -0200
- Subject: Re: [PATCH v2] aarch64: Add split-stack initial support
- Authentication-results: sourceware.org; auth=none
- References: <AM5PR0802MB261081EB50235D83C6462C4483DF0@AM5PR0802MB2610.eurprd08.prod.outlook.com>
On 14/10/2016 15:59, Wilco Dijkstra wrote:
> Hi,
>
Thanks for the thoughtful review and sorry for late response.
>> Split-stack prologue on function entry is as follow (this goes before the
>> usual function prologue):
>
>> mrs x9, tpidr_el0
>> mov x10, -<required stack allocation>
>
> As Jiong already remarked, the nop won't work. Do we know the maximum adjustment
> that the linker is allowed to make? If so, and we can limit the adjustment to 16MB in
> most cases, emitting 2 subtracts is best. Larger offset need mov/movk/sub but that
> should be extremely rare.
There is no limit afaik on gold split stack allocation handling,
and I think one could be added for each backend (in the method
override require to implement it).
In fact it is not really required to tie the nop generation with the
instruction generated by 'aarch64_internal_mov_immediate', it is
just a matter to simplify linker code.
And although 16MB should be rare, nilptr2.go tests allocates 134217824
so this test fails with this low stack limit. I am not sure how well
is the stack usage on 'go', but I think we should at least support
current testcase scenario. So for current iteration I kept my
current approach, but I am open to suggestions.
>
>> nop/movk
>
>> add x10, sp, x10
>> ldr x9, [x9, 16]
>
> Is there any need to detect underflow of x10 or is there a guarantee that stacks are
> never allocated in the low 2GB (given the maximum adjustment is 2GB)? It's safe
> to do a signed comparison.
I do not think so, at least none of current backend that implements
split stack do so.
>
>> cmp x10, x9
>> b.cs enough
>
> Why save/restore x30 and the call x30+8 trick when we could pass the
> continuation address and use a tailcall? That also avoids emitting extra unwind info.
>
>> stp x30, [sp, -16]
>> bl __morestack
>> ldp x30, [sp], 16
>> ret
>
> This part doesn't make any sense - both x28 and carry flag as an input, and spread
> across the prolog - why???
>
>> enough:
>> mov x10, sp
> [prolog]
>> b.cs continue
>> mov x10, x28
> continue:
> [rest of function]
>
> Why not do this?
>
> function:
> mrs x9, tpidr_el0
> sub x10, sp, N & 0xfff000
> sub x10, x10, N & 0xfff
> ldr x9, [x9, 16]
> adr x12, main_fn_entry
> mov x11, sp [if function has stacked arguments]
> cmp x10, x9
> b.ge main_fn_entry
> b __morestack
> main_fn_entry: [x11 is argument pointer]
> [prolog]
> [rest of function]
>
> In __morestack you need to save x8 as well (another argument register!) and x12 (the
> continuation address). After returning from the call x8 doesn't need to be preserved.
Indeed this strategy is way better and I adjusted the code follow it.
The only change is I am using a:
[...]
cmp x9, x10
b.lt main_fn_entr
b __morestack.
[...]
So I can issue a 'cmp <continuation address>, 0' on __morestack to indicate
the function was called.
>
> There are several issues with unwinding in __morestack. x28 is not described as a callee-save
> so will be corrupted if unwinding across a __morestack call. This won't unwind correctly after
> the ldp as the unwinder will use the restored frame pointer to try to restore x29/x30:
>
> + ldp x29, x30, [x28, STACKFRAME_BASE]
> + ldr x28, [x28, STACKFRAME_BASE + 80]
> +
> + .cfi_remember_state
> + .cfi_restore 30
> + .cfi_restore 29
> + .cfi_def_cfa 31, 0
Indeed, it misses x28 save/restore. I think I have added the missing bits, but I
must confess that I am not well versed in CFI directives. I will appreciate if
you could help me on this new version.
>
> This stores a random x30 value on the stack, what is the purpose of this? Nothing can unwind
> to here:
>
> + # Start using new stack
> + stp x29, x30, [x0, -16]!
> + mov sp, x0
>
> Also we no longer need split_stack_arg_pointer_used_p () or any code that uses it (functions
> that don't have any arguments passed on the stack could omit the mov x11, sp).
Right, we new strategy you proposed to do a branch this is indeed not
really required. I remove it from on this new patch.
>
> Wilco
>
From dd2927aa5deb8d609c748014f3b566962fb852c5 Mon Sep 17 00:00:00 2001
From: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Date: Wed, 4 May 2016 21:13:39 +0000
Subject: [PATCH 2/2] aarch64: Add split-stack initial support
This patch adds the split-stack support on aarch64 (PR #67877). As for
other ports this patch should be used along with glibc and gold support.
The support is done similar to other architectures: a __private_ss field is
added on TCB in glibc, a target-specific __morestack implementation and
helper functions are added in libgcc and compiler supported in adjusted
(split-stack prologue, va_start for argument handling). I also plan to
send the gold support to adjust stack allocation acrosss split-stack
and default code calls.
Current approach is similar to powerpc one: at most 2 GB of stack allocation
is support so stack adjustments can be done with 2 instructions (either just
a movn plus nop or a movn followed by movk). The morestack call is non
standard with x10 hollding the requested stack pointer, x11 the argument
pointer, and x12 to return continuation address. Unwinding is handled by a
personality routine that knows how to find stack segments.
Split-stack prologue on function entry is as follow (this goes before the
usual function prologue):
function:
mrs x9, tpidr_el0
ldr x9, [x9, 16]
mov x10, -<required stack allocation>
nop/movk
add x10, sp, x10
mov x11, sp # if function has stacked arguments
adrp x12, main_fn_entry
add x12, x12, :lo12:.L2
cmp x9, x10
blt cond
b __morestack
cond:
main_fn_entry: # x11 is the argument pointer
[function prologue]
blt cond2 # if function has stacked arguments
mov x11, x28
cond2:
Notes:
1. Even if a function does not allocate a stack frame, a split-stack prologue
is created. It is to avoid issues with tail call for external symbols
which might require linker adjustment (libgo/runtime/go-varargs.c).
2. Basic-block reordering (enabled with -O2) will move split-stack TCB ldr
to after the required stack calculation.
3. Similar to powerpc, When the linker detects a call from split-stack to
non-split-stack code, it adds 16k (or more) to the value found in "allocate"
instructions (so non-split-stack code gets a larger stack). The amount is
tunable by a linker option. The edit means aarch64 does not need to
implement __morestack_non_split, necessary on x86 because insufficient
space is available there to edit the stack comparison code. This feature
is only implemented in the GNU gold linker.
4. AArch64 does not handle >2G stack initially and although it is possible
to implement it, limiting to 2G allows to materize the allocation with
only 2 instructions (mov + movk) and thus simplifying the linker
adjustments required. Supporting multiple threads each requiring more
than 2G of stack is probably not that important, and likely to OOM at
run time.
5. The TCB support on GLIBC is meant to be included in version 2.25.
6. The continuation address materialized on x12 is done using 'adrp'
plus add and a static relocation. Current code uses the
aarch64_expand_mov_immediate function and since a better alternative
would be 'adp', it could be a future optimization (not implemented
in this patch).
libgcc/ChangeLog:
* libgcc/config.host: Use t-stack and t-statck-aarch64 for
aarch64*-*-linux.
* libgcc/config/aarch64/morestack-c.c: New file.
* libgcc/config/aarch64/morestack.S: Likewise.
* libgcc/config/aarch64/t-stack-aarch64: Likewise.
* libgcc/generic-morestack.c (__splitstack_find): Add aarch64-specific
code.
gcc/ChangeLog:
* common/config/aarch64/aarch64-common.c
(aarch64_supports_split_stack): New function.
(TARGET_SUPPORTS_SPLIT_STACK): New macro.
* gcc/config/aarch64/aarch64-linux.h (TARGET_ASM_FILE_END): Remove
macro.
* gcc/config/aarch64/aarch64-protos.h: Add
aarch64_expand_split_stack_prologue and
aarch64_split_stack_space_check.
* gcc/config/aarch64/aarch64.c (split_stack_arg_pointer_used_p): New
function.
(aarch64_expand_prologue): Setup the argument pointer (x10) for
split-stack.
(aarch64_expand_builtin_va_start): Use internal argument pointer
instead of virtual_incoming_args_rtx.
(morestack_ref): New symbol.
(aarch64_expand_split_stack_prologue): New function.
(aarch64_file_end): Emit the split-stack note sections.
(aarch64_internal_arg_pointer): Likewise.
(aarch64_live_on_entry): Set the argument pointer for split-stack.
(aarch64_split_stack_space_check): Likewise.
(TARGET_ASM_FILE_END): New macro.
(TARGET_EXTRA_LIVE_ON_ENTRY): Likewise.
(TARGET_INTERNAL_ARG_POINTER): Likewise.
* gcc/config/aarch64/aarch64.h (aarch64_frame): Add
split_stack_arg_pointer to setup the argument pointer when using
split-stack.
* gcc/config/aarch64/aarch64.md (UNSPEC_STACK_CHECK): New unspec.
(UNSPECV_SPLIT_STACK_RETURN): Likewise.
(split_stack_prologue): New expand.
(split_stack_return): New insn.
(split_stack_space_check): New expand.
* gcc/testsuite/gcc.dg/split-3.c (down): Call va_end after va_start.
* gcc/testsuite/gcc.dg/split-6.c (down): Likewise.
---
gcc/ChangeLog | 37 ++++
gcc/common/config/aarch64/aarch64-common.c | 16 +-
gcc/config/aarch64/aarch64-linux.h | 2 -
gcc/config/aarch64/aarch64-protos.h | 2 +
gcc/config/aarch64/aarch64.c | 218 ++++++++++++++++++++++-
gcc/config/aarch64/aarch64.h | 3 +
gcc/config/aarch64/aarch64.md | 57 ++++++
libgcc/ChangeLog | 11 ++
libgcc/config.host | 1 +
libgcc/config/aarch64/morestack-c.c | 95 ++++++++++
libgcc/config/aarch64/morestack.S | 269 +++++++++++++++++++++++++++++
libgcc/config/aarch64/t-stack-aarch64 | 3 +
libgcc/generic-morestack.c | 1 +
13 files changed, 710 insertions(+), 5 deletions(-)
create mode 100644 libgcc/config/aarch64/morestack-c.c
create mode 100644 libgcc/config/aarch64/morestack.S
create mode 100644 libgcc/config/aarch64/t-stack-aarch64
diff --git a/gcc/ChangeLog b/gcc/ChangeLog
index 34e88da..402ac7c 100644
--- a/gcc/ChangeLog
+++ b/gcc/ChangeLog
@@ -1,3 +1,40 @@
+2016-11-07 Adhemerval Zanella <adhemerval.zanella@linaro.org>
+
+ PR target/67877
+ * common/config/aarch64/aarch64-common.c
+ (aarch64_supports_split_stack): New function.
+ (TARGET_SUPPORTS_SPLIT_STACK): New macro.
+ * gcc/config/aarch64/aarch64-linux.h (TARGET_ASM_FILE_END): Remove
+ macro.
+ * gcc/config/aarch64/aarch64-protos.h: Add
+ aarch64_expand_split_stack_prologue and
+ aarch64_split_stack_space_check.
+ * gcc/config/aarch64/aarch64.c (split_stack_arg_pointer_used_p): New
+ function.
+ (aarch64_expand_prologue): Setup the argument pointer (x10) for
+ split-stack.
+ (aarch64_expand_builtin_va_start): Use internal argument pointer
+ instead of virtual_incoming_args_rtx.
+ (morestack_ref): New symbol.
+ (aarch64_expand_split_stack_prologue): New function.
+ (aarch64_file_end): Emit the split-stack note sections.
+ (aarch64_internal_arg_pointer): Likewise.
+ (aarch64_live_on_entry): Set the argument pointer for split-stack.
+ (aarch64_split_stack_space_check): Likewise.
+ (TARGET_ASM_FILE_END): New macro.
+ (TARGET_EXTRA_LIVE_ON_ENTRY): Likewise.
+ (TARGET_INTERNAL_ARG_POINTER): Likewise.
+ * gcc/config/aarch64/aarch64.h (aarch64_frame): Add
+ split_stack_arg_pointer to setup the argument pointer when using
+ split-stack.
+ * gcc/config/aarch64/aarch64.md (UNSPEC_STACK_CHECK): New unspec.
+ (UNSPECV_SPLIT_STACK_RETURN): Likewise.
+ (split_stack_prologue): New expand.
+ (split_stack_return): New insn.
+ (split_stack_space_check): New expand.
+ * gcc/testsuite/gcc.dg/split-3.c (down): Call va_end after va_start.
+ * gcc/testsuite/gcc.dg/split-6.c (down): Likewise.
+
2016-11-07 Jakub Jelinek <jakub@redhat.com>
PR target/77834
diff --git a/gcc/common/config/aarch64/aarch64-common.c b/gcc/common/config/aarch64/aarch64-common.c
index 90f5f6b..7641279 100644
--- a/gcc/common/config/aarch64/aarch64-common.c
+++ b/gcc/common/config/aarch64/aarch64-common.c
@@ -107,6 +107,21 @@ aarch64_handle_option (struct gcc_options *opts,
}
}
+/* -fsplit-stack uses a TCB field available on glibc-2.25. GLIBC also
+ exports symbol, __tcb_private_ss, to signal it has the field available
+ on TCB allocation. This aims to prevent binaries linked against newer
+ GLIBC to run on non-supported ones. */
+
+static bool
+aarch64_supports_split_stack (bool report ATTRIBUTE_UNUSED,
+ struct gcc_options *opts ATTRIBUTE_UNUSED)
+{
+ return true;
+}
+
+#undef TARGET_SUPPORTS_SPLIT_STACK
+#define TARGET_SUPPORTS_SPLIT_STACK aarch64_supports_split_stack
+
struct gcc_targetm_common targetm_common = TARGETM_COMMON_INITIALIZER;
/* An ISA extension in the co-processor and main instruction set space. */
@@ -340,4 +355,3 @@ aarch64_rewrite_mcpu (int argc, const char **argv)
}
#undef AARCH64_CPU_NAME_LENGTH
-
diff --git a/gcc/config/aarch64/aarch64-linux.h b/gcc/config/aarch64/aarch64-linux.h
index 5fcaa59..ab3208b 100644
--- a/gcc/config/aarch64/aarch64-linux.h
+++ b/gcc/config/aarch64/aarch64-linux.h
@@ -80,8 +80,6 @@
} \
while (0)
-#define TARGET_ASM_FILE_END file_end_indicate_exec_stack
-
/* Uninitialized common symbols in non-PIE executables, even with
strong definitions in dependent shared libraries, will resolve
to COPY relocated symbol in the executable. See PR65780. */
diff --git a/gcc/config/aarch64/aarch64-protos.h b/gcc/config/aarch64/aarch64-protos.h
index 4f4989d..60e0c99 100644
--- a/gcc/config/aarch64/aarch64-protos.h
+++ b/gcc/config/aarch64/aarch64-protos.h
@@ -373,6 +373,8 @@ void aarch64_err_no_fpadvsimd (machine_mode, const char *);
void aarch64_expand_epilogue (bool);
void aarch64_expand_mov_immediate (rtx, rtx);
void aarch64_expand_prologue (void);
+void aarch64_expand_split_stack_prologue (void);
+void aarch64_split_stack_space_check (rtx, rtx);
void aarch64_expand_vector_init (rtx, rtx);
void aarch64_init_cumulative_args (CUMULATIVE_ARGS *, const_tree, rtx,
const_tree, unsigned);
diff --git a/gcc/config/aarch64/aarch64.c b/gcc/config/aarch64/aarch64.c
index b7d4640..ba1a8da 100644
--- a/gcc/config/aarch64/aarch64.c
+++ b/gcc/config/aarch64/aarch64.c
@@ -895,7 +895,12 @@ aarch64_gen_far_branch (rtx * operands, int pos_label, const char * dest,
snprintf (buffer, sizeof (buffer), "%s%s", branch_format, label_ptr);
output_asm_insn (buffer, operands);
- snprintf (buffer, sizeof (buffer), "b\t%%l%d\n%s:", pos_label, label_ptr);
+ if (GET_CODE (operands[pos_label]) == LABEL_REF)
+ snprintf (buffer, sizeof (buffer), "b\t%%l%d\n%s:", pos_label,
+ label_ptr);
+ else
+ snprintf (buffer, sizeof (buffer), "b\t%%%d\n%s:", pos_label,
+ label_ptr);
operands[pos_label] = dest_label;
output_asm_insn (buffer, operands);
return "";
@@ -3223,6 +3228,24 @@ aarch64_restore_callee_saves (machine_mode mode,
}
}
+/* Return whether the split-stack arg pointer (x10) is used. */
+
+static bool
+split_stack_arg_pointer_used_p (void)
+{
+ bool using_split_stack = (flag_split_stack
+ && (lookup_attribute ("no_split_stack",
+ DECL_ATTRIBUTES (cfun->decl))
+ == NULL));
+ if (using_split_stack == false)
+ return false;
+
+ /* Function uses stacked arguments. */
+ return (cfun->machine->frame.saved_regs_size
+ || cfun->machine->frame.saved_varargs_size);
+}
+
+
/* AArch64 stack frames generated by this compiler look like:
+-------------------------------+
@@ -3316,6 +3339,28 @@ aarch64_expand_prologue (void)
aarch64_save_callee_saves (DFmode, callee_offset, V0_REGNUM, V31_REGNUM,
callee_adjust != 0 || frame_pointer_needed);
aarch64_sub_sp (IP1_REGNUM, final_adjust, !frame_pointer_needed);
+
+ if (split_stack_arg_pointer_used_p ())
+ {
+ /* Setup the argument pointer (x10) for -fsplit-stack code. If
+ __morestack was called, it will left the arg pointer to the
+ old stack in x28. Otherwise, the argument pointer is the top
+ of current frame. */
+ rtx x11 = gen_rtx_REG (Pmode, R11_REGNUM);
+ rtx x28 = gen_rtx_REG (Pmode, R28_REGNUM);
+ rtx cc_reg = gen_rtx_REG (CCmode, CC_REGNUM);
+
+ rtx not_more = gen_label_rtx ();
+
+ rtx cmp = gen_rtx_fmt_ee (LT, VOIDmode, cc_reg, const0_rtx);
+ rtx jump = emit_jump_insn (gen_condjump (cmp, cc_reg, not_more));
+ JUMP_LABEL (jump) = not_more;
+ LABEL_NUSES (not_more) += 1;
+
+ emit_move_insn (x11, x28);
+
+ emit_label (not_more);
+ }
}
/* Return TRUE if we can use a simple_return insn.
@@ -9791,7 +9836,7 @@ aarch64_expand_builtin_va_start (tree valist, rtx nextarg ATTRIBUTE_UNUSED)
/* Emit code to initialize STACK, which points to the next varargs stack
argument. CUM->AAPCS_STACK_SIZE gives the number of stack words used
by named arguments. STACK is 8-byte aligned. */
- t = make_tree (TREE_TYPE (stack), virtual_incoming_args_rtx);
+ t = make_tree (TREE_TYPE (stack), crtl->args.internal_arg_pointer);
if (cum->aapcs_stack_size > 0)
t = fold_build_pointer_plus_hwi (t, cum->aapcs_stack_size * UNITS_PER_WORD);
t = build2 (MODIFY_EXPR, TREE_TYPE (stack), stack, t);
@@ -14168,6 +14213,166 @@ aarch64_optab_supported_p (int op, machine_mode mode1, machine_mode,
}
}
+/* -fsplit-stack support. */
+
+/* A SYMBOL_REF for __morestack. */
+static GTY(()) rtx morestack_ref;
+
+/* Emit -fsplit-stack prologue, which goes before the regular function
+ prologue. */
+
+void
+aarch64_expand_split_stack_prologue (void)
+{
+ rtx ssvalue, mem;
+ rtx reg10, reg11, reg12, cc, cmp, jump;
+ HOST_WIDE_INT allocate;
+ rtx_code_label *ok_label = NULL;
+ /* Offset from thread pointer to __private_ss. */
+ int psso = 0x10;
+
+ gcc_assert (flag_split_stack && reload_completed);
+
+ /* It limits total maximum stack allocation on 2G so its value can be
+ materialized using two instructions at most (movn/movk). It might be
+ used by the linker to add some extra space for split calling non split
+ stack functions. */
+ allocate = cfun->machine->frame.frame_size;
+ if (allocate > ((HOST_WIDE_INT) 1 << 31))
+ {
+ sorry ("Stack frame larger than 2G is not supported for -fsplit-stack");
+ return;
+ }
+
+ if (morestack_ref == NULL_RTX)
+ {
+ morestack_ref = gen_rtx_SYMBOL_REF (Pmode, "__morestack");
+ SYMBOL_REF_FLAGS (morestack_ref) |= (SYMBOL_FLAG_LOCAL
+ | SYMBOL_FLAG_FUNCTION);
+ }
+
+ /* Load __private_ss from TCB. */
+ ssvalue = gen_rtx_REG (Pmode, R9_REGNUM);
+ emit_insn (gen_aarch64_load_tp_hard (ssvalue));
+ mem = gen_rtx_MEM (Pmode, plus_constant (Pmode, ssvalue, psso));
+ emit_move_insn (ssvalue, mem);
+
+ /* Always emit two insns to calculate the requested stack, so the linker
+ can edit them when adjusting size for calling non-split-stack code. */
+ reg10 = gen_rtx_REG (Pmode, R10_REGNUM);
+ int ninsn = aarch64_internal_mov_immediate (reg10, GEN_INT (-allocate),
+ true, Pmode);
+ gcc_assert (ninsn == 1 || ninsn == 2);
+ if (ninsn == 1)
+ emit_insn (gen_nop ());
+ emit_insn (gen_add3_insn (reg10, stack_pointer_rtx, reg10));
+
+ ok_label = gen_label_rtx ();
+
+ /* If function uses stacked arguments save the old stack value so morestack
+ can return it. */
+ reg11 = gen_rtx_REG (Pmode, R11_REGNUM);
+ if (cfun->machine->frame.saved_regs_size
+ || cfun->machine->frame.saved_varargs_size)
+ emit_move_insn (reg11, stack_pointer_rtx);
+
+ /* x12 holds the continuation address used to return to function. */
+ reg12 = gen_rtx_REG (Pmode, R12_REGNUM);
+ aarch64_expand_mov_immediate (reg12, gen_rtx_LABEL_REF (VOIDmode, ok_label));
+
+ /* Jump to __morestack call if current __private_ss is not suffice. */
+ cc = aarch64_gen_compare_reg (GE, ssvalue, reg10);
+ cmp = gen_rtx_fmt_ee (GE, VOIDmode, cc, const0_rtx);
+ jump = gen_split_stack_cond_call (morestack_ref, cmp, ok_label, reg12);
+
+ aarch64_emit_unlikely_jump (jump);
+ JUMP_LABEL (jump) = ok_label;
+ LABEL_NUSES (ok_label)++;
+
+ /* __morestack will call us here. */
+ emit_label (ok_label);
+}
+
+/* Implement TARGET_ASM_FILE_END. */
+
+static void
+aarch64_file_end (void)
+{
+ file_end_indicate_exec_stack ();
+
+ if (flag_split_stack)
+ file_end_indicate_split_stack ();
+}
+
+/* Return the internal arg pointer used for function incoming arguments. */
+
+static rtx
+aarch64_internal_arg_pointer (void)
+{
+ if (flag_split_stack
+ && (lookup_attribute ("no_split_stack", DECL_ATTRIBUTES (cfun->decl))
+ == NULL))
+ {
+ if (cfun->machine->frame.split_stack_arg_pointer == NULL_RTX)
+ {
+ rtx pat;
+
+ cfun->machine->frame.split_stack_arg_pointer = gen_reg_rtx (Pmode);
+ REG_POINTER (cfun->machine->frame.split_stack_arg_pointer) = 1;
+
+ /* Put the pseudo initialization right after the note at the
+ beginning of the function. */
+ pat = gen_rtx_SET (cfun->machine->frame.split_stack_arg_pointer,
+ gen_rtx_REG (Pmode, R11_REGNUM));
+ push_topmost_sequence ();
+ emit_insn_after (pat, get_insns ());
+ pop_topmost_sequence ();
+ }
+ return plus_constant (Pmode, cfun->machine->frame.split_stack_arg_pointer,
+ FIRST_PARM_OFFSET (current_function_decl));
+ }
+ return virtual_incoming_args_rtx;
+}
+
+/* We may have to tell the dataflow pass that the split stack prologue
+ is initializing a register. */
+
+static void
+aarch64_live_on_entry (bitmap regs)
+{
+ if (flag_split_stack)
+ bitmap_set_bit (regs, R11_REGNUM);
+}
+
+/* Emit -fsplit-stack dynamic stack allocation space check. */
+
+void
+aarch64_split_stack_space_check (rtx size, rtx label)
+{
+ rtx mem, ssvalue, cc, cmp, jump, temp;
+ rtx requested = gen_reg_rtx (Pmode);
+ /* Offset from thread pointer to __private_ss. */
+ int psso = 0x10;
+
+ /* Load __private_ss from TCB. */
+ ssvalue = gen_rtx_REG (Pmode, R9_REGNUM);
+ emit_insn (gen_aarch64_load_tp_hard (ssvalue));
+ mem = gen_rtx_MEM (Pmode, plus_constant (Pmode, ssvalue, psso));
+ emit_move_insn (ssvalue, mem);
+
+ temp = gen_rtx_REG (Pmode, R10_REGNUM);
+
+ /* And compare it with frame pointer plus required stack. */
+ size = force_reg (Pmode, size);
+ emit_move_insn (requested, gen_rtx_MINUS (Pmode, stack_pointer_rtx, size));
+
+ /* Jump to __morestack call if current __private_ss is not suffice. */
+ cc = aarch64_gen_compare_reg (LT, temp, ssvalue);
+ cmp = gen_rtx_fmt_ee (GEU, VOIDmode, cc, const0_rtx);
+ jump = emit_jump_insn (gen_condjump (cmp, cc, label));
+ JUMP_LABEL (jump) = label;
+}
+
#undef TARGET_ADDRESS_COST
#define TARGET_ADDRESS_COST aarch64_address_cost
@@ -14194,6 +14399,9 @@ aarch64_optab_supported_p (int op, machine_mode mode1, machine_mode,
#undef TARGET_ASM_FILE_START
#define TARGET_ASM_FILE_START aarch64_start_file
+#undef TARGET_ASM_FILE_END
+#define TARGET_ASM_FILE_END aarch64_file_end
+
#undef TARGET_ASM_OUTPUT_MI_THUNK
#define TARGET_ASM_OUTPUT_MI_THUNK aarch64_output_mi_thunk
@@ -14276,6 +14484,12 @@ aarch64_optab_supported_p (int op, machine_mode mode1, machine_mode,
#undef TARGET_FRAME_POINTER_REQUIRED
#define TARGET_FRAME_POINTER_REQUIRED aarch64_frame_pointer_required
+#undef TARGET_EXTRA_LIVE_ON_ENTRY
+#define TARGET_EXTRA_LIVE_ON_ENTRY aarch64_live_on_entry
+
+#undef TARGET_INTERNAL_ARG_POINTER
+#define TARGET_INTERNAL_ARG_POINTER aarch64_internal_arg_pointer
+
#undef TARGET_GIMPLE_FOLD_BUILTIN
#define TARGET_GIMPLE_FOLD_BUILTIN aarch64_gimple_fold_builtin
diff --git a/gcc/config/aarch64/aarch64.h b/gcc/config/aarch64/aarch64.h
index 584ff5c..a464c66 100644
--- a/gcc/config/aarch64/aarch64.h
+++ b/gcc/config/aarch64/aarch64.h
@@ -586,6 +586,9 @@ struct GTY (()) aarch64_frame
unsigned wb_candidate2;
bool laid_out;
+
+ /* Alternative internal arg pointer for -fsplit-stack. */
+ rtx split_stack_arg_pointer;
};
typedef struct GTY (()) machine_function
diff --git a/gcc/config/aarch64/aarch64.md b/gcc/config/aarch64/aarch64.md
index 46eaa30..7e49faf 100644
--- a/gcc/config/aarch64/aarch64.md
+++ b/gcc/config/aarch64/aarch64.md
@@ -130,6 +130,7 @@
UNSPEC_VSTRUCTDUMMY
UNSPEC_SP_SET
UNSPEC_SP_TEST
+ UNSPEC_STACK_CHECK
UNSPEC_RSQRT
UNSPEC_RSQRTE
UNSPEC_RSQRTS
@@ -144,6 +145,7 @@
UNSPECV_SET_FPSR ; Represent assign of FPSR content.
UNSPECV_BLOCKAGE ; Represent a blockage
UNSPECV_PROBE_STACK_RANGE ; Represent stack range probing.
+ UNSPECV_SPLIT_STACK_CALL ; Represent a morestack call
]
)
@@ -5390,3 +5392,58 @@
;; ldp/stp peephole patterns
(include "aarch64-ldpstp.md")
+
+;; Handle -fsplit-stack
+(define_expand "split_stack_prologue"
+ [(const_int 0)]
+ ""
+{
+ aarch64_expand_split_stack_prologue ();
+ DONE;
+})
+
+;; If there are operand 0 bytes available on the stack, jump to
+;; operand 1.
+(define_expand "split_stack_space_check"
+ [(set (match_dup 2) (compare:CC (match_dup 3) (match_dup 2)))
+ (set (pc) (if_then_else
+ (geu (match_dup 4) (const_int 0))
+ (label_ref (match_operand 1))
+ (pc)))]
+ ""
+{
+ aarch64_split_stack_space_check (operands[0], operands[1]);
+ DONE;
+})
+
+;; A __morestack call using branch
+
+(define_expand "split_stack_cond_call"
+ [(match_operand 0 "aarch64_call_insn_operand" "")
+ (match_operand 1 "" "")
+ (match_operand 2 "" "")
+ (match_operand 3 "" "")]
+ ""
+{
+ emit_jump_insn (gen_split_stack_cond_call_di (operands[0], operands[1],
+ operands[2], operands[3]));
+ DONE;
+})
+
+
+(define_insn "split_stack_cond_call_<mode>"
+ [(set (pc)
+ (if_then_else
+ (match_operand 1 "" "")
+ (label_ref (match_operand 2 "" ""))
+ (pc)))
+ (set (reg:P 1) (unspec_volatile [(match_operand 0 "aarch64_call_insn_operand" "")
+ (reg:P 1)]
+ UNSPECV_SPLIT_STACK_CALL))
+ (use (match_operand 3 "" ""))]
+ ""
+ {
+ return aarch64_gen_far_branch (operands, 0, "Lbcond", "b%M1\\t");
+ }
+ [(set_attr "type" "branch")]
+)
diff --git a/libgcc/ChangeLog b/libgcc/ChangeLog
index e3a81f6..075853e 100644
--- a/libgcc/ChangeLog
+++ b/libgcc/ChangeLog
@@ -1,3 +1,14 @@
+2016-11-07 Adhemerval Zanella <adhemerval.zanella@linaro.org>
+
+ PR target/67877
+ * libgcc/config.host: Use t-stack and t-statck-aarch64 for
+ aarch64*-*-linux.
+ * libgcc/config/aarch64/morestack-c.c: New file.
+ * libgcc/config/aarch64/morestack.S: Likewise.
+ * libgcc/config/aarch64/t-stack-aarch64: Likewise.
+ * libgcc/generic-morestack.c (__splitstack_find): Add aarch64-specific
+ code.
+
2016-11-03 Martin Liska <mliska@suse.cz>
* libgcov-profiler.c (__gcov_time_profiler): Remove.
diff --git a/libgcc/config.host b/libgcc/config.host
index 4291cfe..08234fc 100644
--- a/libgcc/config.host
+++ b/libgcc/config.host
@@ -341,6 +341,7 @@ aarch64*-*-linux*)
md_unwind_header=aarch64/linux-unwind.h
tmake_file="${tmake_file} ${cpu_type}/t-aarch64"
tmake_file="${tmake_file} ${cpu_type}/t-softfp t-softfp t-crtfm"
+ tmake_file="${tmake_file} t-stack aarch64/t-stack-aarch64"
;;
alpha*-*-linux*)
tmake_file="${tmake_file} alpha/t-alpha alpha/t-ieee t-crtfm alpha/t-linux"
diff --git a/libgcc/config/aarch64/morestack-c.c b/libgcc/config/aarch64/morestack-c.c
new file mode 100644
index 0000000..8df7895
--- /dev/null
+++ b/libgcc/config/aarch64/morestack-c.c
@@ -0,0 +1,95 @@
+/* AArch64 support for -fsplit-stack.
+ * Copyright (C) 2016 Free Software Foundation, Inc.
+ *
+ * This file is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 3, or (at your option) any
+ * later version.
+ *
+ * This file is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * Under Section 7 of GPL version 3, you are granted additional
+ * permissions described in the GCC Runtime Library Exception, version
+ * 3.1, as published by the Free Software Foundation.
+ *
+ * You should have received a copy of the GNU General Public License and
+ * a copy of the GCC Runtime Library Exception along with this program;
+ * see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
+ * <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef inhibit_libc
+
+#include <stdint.h>
+#include <stdlib.h>
+#include <stddef.h>
+#include "generic-morestack.h"
+
+/* This is based on GLIBC definition (version 2.24). There is no need to
+ keep it sync since new fields are added on the end of structure and do
+ not change the '__private_ss' layout. */
+typedef struct
+{
+ void *dtv;
+ void *private;
+ void *__private_ss;
+} tcbhead_t;
+
+#define INITIAL_STACK_SIZE 0x4000
+#define BACKOFF 0x1000
+
+void __generic_morestack_set_initial_sp (void *sp, size_t len);
+void *__morestack_get_guard (void);
+void __morestack_set_guard (void *);
+void *__morestack_make_guard (void *stack, size_t size);
+void __morestack_load_mmap (void);
+
+/* We declare is as weak so it fails either at stack linking or
+ at runtime if the GLIBC does not have the required TCB field. */
+extern void __tcb_private_ss (void) __attribute__ ((weak));
+
+/* Initialize the stack guard when the program starts or when a new
+ thread. This is called from a constructor using ctors section. */
+void
+__stack_split_initialize (void)
+{
+ __tcb_private_ss ();
+
+ register void* sp __asm__ ("sp");
+ tcbhead_t *tcb = ((tcbhead_t *) __builtin_thread_pointer ());
+ tcb->__private_ss = (void*)((uintptr_t)sp - INITIAL_STACK_SIZE);
+ return __generic_morestack_set_initial_sp (sp, INITIAL_STACK_SIZE);
+}
+
+/* Return current __private_ss. */
+void *
+__morestack_get_guard (void)
+{
+ tcbhead_t *tcb = ((tcbhead_t *) __builtin_thread_pointer ());
+ return tcb->__private_ss;
+}
+
+/* Set __private_ss to ptr. */
+void
+__morestack_set_guard (void *ptr)
+{
+ tcbhead_t *tcb = ((tcbhead_t *) __builtin_thread_pointer ());
+ tcb->__private_ss = ptr;
+}
+
+/* Return the stack guard value for given stack. */
+void *
+__morestack_make_guard (void *stack, size_t size)
+{
+ return (void*)((uintptr_t)stack - size + BACKOFF);
+}
+
+/* Make __stack_split_initialize a high priority constructor. */
+static void (*const ctors [])
+ __attribute__ ((used, section (".ctors.65535"), aligned (sizeof (void *))))
+ = { __stack_split_initialize, __morestack_load_mmap };
+
+#endif /* !defined (inhibit_libc) */
diff --git a/libgcc/config/aarch64/morestack.S b/libgcc/config/aarch64/morestack.S
new file mode 100644
index 0000000..8b68c08
--- /dev/null
+++ b/libgcc/config/aarch64/morestack.S
@@ -0,0 +1,269 @@
+# AArch64 support for -fsplit-stack.
+# Copyright (C) 2016 Free Software Foundation, Inc.
+
+# This file is part of GCC.
+
+# GCC is free software; you can redistribute it and/or modify it under
+# the terms of the GNU General Public License as published by the Free
+# Software Foundation; either version 3, or (at your option) any later
+# version.
+
+# GCC is distributed in the hope that it will be useful, but WITHOUT ANY
+# WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
+# for more details.
+
+# Under Section 7 of GPL version 3, you are granted additional
+# permissions described in the GCC Runtime Library Exception, version
+# 3.1, as published by the Free Software Foundation.
+
+# You should have received a copy of the GNU General Public License and
+# a copy of the GCC Runtime Library Exception along with this program;
+# see the files COPYING3 and COPYING.RUNTIME respectively. If not, see
+# <http://www.gnu.org/licenses/>.
+
+/* Define an entry point visible from C. */
+#define ENTRY(name) \
+ .globl name; \
+ .type name,%function; \
+ .align 4; \
+ name##:
+
+#define END(name) \
+ .size name,.-name
+
+
+#define MORESTACK_FRAMESIZE 128
+/* Offset based on function stack to get its argument from __morestack
+ frame. */
+#define STACKFRAME_BASE (-MORESTACK_FRAMESIZE)
+/* Offset from __morestack frame where the new stack size is saved and
+ passed to __generic_morestack. */
+#define NEWSTACK_SAVE 88
+/* Offset from __morestack frame where the arguments size saved and
+ passed to __generic_morestack. */
+#define ARGS_SIZE_SAVE 80
+
+#define BACKOFF 0x2000
+# Large excess allocated when calling non-split-stack code.
+#define NON_SPLIT_STACK 0x100000
+
+# TCB offset of __private_ss
+#define TCB_PRIVATE_SS #16
+
+ .text
+ENTRY(__morestack_non_split)
+ .cfi_startproc
+# We use a cleanup to restore the tcbhead_t.__private_ss if
+# an exception is thrown through this code.
+ add x10, x10, NON_SPLIT_STACK
+ .cfi_endproc
+END(__morestack_non_split)
+# Fall through into __morestack
+
+# This function is called with non-standard calling conventions, on entry
+# x10 is the requested stack pointer. The split-stack prologue is in the
+# form:
+#
+# function:
+# mrs x9, tpidr_el0
+# sub x10, sp, N & 0xfff000
+# sub x10, x10, N & 0xfff
+# ldr x9, [x9, 16]
+# adrp x12, main_fn_entry
+# add x12, x12, :lo12:main_fn_entry
+# [mov x11, sp] # If function has stacked arguments
+# cmp x9, x10
+# b.lt main_fn_entry
+# b __morestack
+# main_fn_entry: [x11 is argument pointer]
+# [prolog]
+#
+# The normal function prologue follows here, with a small addition at the
+# end to set up the argument pointer if required (the prolog):
+#
+# [...] # default function prologue
+# b.lt function:
+# mov x11, x28
+# function:
+#
+#
+# The N bit is also restored to indicate that the function is called
+# (so the prologue addition can set up the argument pointer correctly).
+
+ENTRY(__morestack)
+.LFB1:
+ .cfi_startproc
+
+#ifdef __PIC__
+ .cfi_personality 0x9b,DW.ref.__gcc_personality_v0
+ .cfi_lsda 0x1b,.LLSDA1
+#else
+ .cfi_personality 0x3,__gcc_personality_v0
+ .cfi_lsda 0x3,.LLSDA1
+#endif
+
+ # Calculate requested stack size.
+ sub x10, sp, x10
+ # Save parameters
+ stp x29, x30, [sp, -MORESTACK_FRAMESIZE]!
+ .cfi_def_cfa_offset MORESTACK_FRAMESIZE
+ .cfi_offset 29, -MORESTACK_FRAMESIZE
+ .cfi_offset 30, -MORESTACK_FRAMESIZE+8
+ add x29, sp, 0
+ .cfi_def_cfa_register 29
+ # Adjust the requested stack size for the frame pointer save.
+ stp x0, x1, [sp, 16]
+ stp x2, x3, [sp, 32]
+ add x10, x10, BACKOFF
+ stp x4, x5, [sp, 48]
+ stp x6, x7, [sp, 64]
+ stp x8, x10, [sp, 80]
+ stp x11, x12, [sp, 96]
+ str x28, [sp, 112]
+ .cfi_offset 28, -112
+
+ # Setup on x28 the function initial frame pointer.
+ add x28, sp, MORESTACK_FRAMESIZE
+
+ # void __morestack_block_signals (void)
+ bl __morestack_block_signals
+
+ # void *__generic_morestack (size_t *pframe_size,
+ # void *old_stack,
+ # size_t param_size)
+ # pframe_size: is the size of the required stack frame (the function
+ # amount of space remaining on the allocated stack).
+ # old_stack: points at the parameters the old stack
+ # param_size: size in bytes of parameters to copy to the new stack.
+ add x0, x28, STACKFRAME_BASE + NEWSTACK_SAVE
+ mov x1, x28
+ mov x2, 0
+ bl __generic_morestack
+
+ # Start using new stack
+ str x29, [x0, -16]!
+ mov sp, x0
+
+ # Set __private_ss stack guard for the new stack.
+ ldr x9, [x28, STACKFRAME_BASE + NEWSTACK_SAVE]
+ add x0, x0, BACKOFF
+ sub x0, x0, 16
+ sub x0, x0, x9
+.LEHB0:
+ mrs x1, tpidr_el0
+ str x0, [x1, TCB_PRIVATE_SS]
+
+ # void __morestack_unblock_signals (void)
+ bl __morestack_unblock_signals
+
+ # Set up for a call to the target function.
+ ldp x0, x1, [x28, STACKFRAME_BASE + 16]
+ ldp x2, x3, [x28, STACKFRAME_BASE + 32]
+ ldp x4, x5, [x28, STACKFRAME_BASE + 48]
+ ldp x6, x7, [x28, STACKFRAME_BASE + 64]
+ ldr x8, [x28, STACKFRAME_BASE + 80]
+ ldp x11, x12, [x28, STACKFRAME_BASE + 96]
+ # Indicate __morestack was called.
+ cmp x12, 0
+ blr x12
+
+ stp x0, x1, [x28, STACKFRAME_BASE + 16]
+ stp x2, x3, [x28, STACKFRAME_BASE + 32]
+ stp x4, x5, [x28, STACKFRAME_BASE + 48]
+ stp x6, x7, [x28, STACKFRAME_BASE + 64]
+
+ bl __morestack_block_signals
+
+ # void *__generic_releasestack (size_t *pavailable)
+ add x0, x28, STACKFRAME_BASE + NEWSTACK_SAVE
+ bl __generic_releasestack
+
+ # Reset __private_ss stack guard to value for old stack
+ ldr x9, [x28, STACKFRAME_BASE + NEWSTACK_SAVE]
+ add x0, x0, BACKOFF
+ sub x0, x0, x9
+
+ # Update TCB split stack field
+.LEHE0:
+ mrs x1, tpidr_el0
+ str x0, [x1, TCB_PRIVATE_SS]
+
+ bl __morestack_unblock_signals
+
+ # Use old stack again.
+ #sub sp, x28, 16
+ mov sp, x28
+
+ ldp x0, x1, [x28, STACKFRAME_BASE + 16]
+ ldp x2, x3, [x28, STACKFRAME_BASE + 32]
+ ldp x4, x5, [x28, STACKFRAME_BASE + 48]
+ ldp x6, x7, [x28, STACKFRAME_BASE + 64]
+ ldp x29, x30, [x28, STACKFRAME_BASE]
+ ldr x28, [x28, STACKFRAME_BASE + 112]
+
+ .cfi_remember_state
+ .cfi_restore 30
+ .cfi_restore 29
+ .cfi_restore 28
+ .cfi_def_cfa 31, 0
+
+ ret
+
+# This is the cleanup code called by the stack unwinder when
+# unwinding through code between .LEHB0 and .LEHE0 above.
+cleanup:
+ .cfi_restore_state
+ str x0, [x28, STACKFRAME_BASE]
+ # size_t __generic_findstack (void *stack)
+ mov x0, x28
+ bl __generic_findstack
+ sub x0, x28, x0
+ add x0, x0, BACKOFF
+ # Restore tcbhead_t.__private_ss
+ mrs x1, tpidr_el0
+ str x0, [x1, TCB_PRIVATE_SS]
+ ldr x0, [x28, STACKFRAME_BASE]
+ b _Unwind_Resume
+ .cfi_endproc
+END(__morestack)
+
+ .section .gcc_except_table,"a",@progbits
+ .align 4
+.LLSDA1:
+ # @LPStart format (omit)
+ .byte 0xff
+ # @TType format (omit)
+ .byte 0xff
+ # Call-site format (uleb128)
+ .byte 0x1
+ # Call-site table length
+ .uleb128 .LLSDACSE1-.LLSDACSB1
+.LLSDACSB1:
+ # region 0 start
+ .uleb128 .LEHB0-.LFB1
+ # length
+ .uleb128 .LEHE0-.LEHB0
+ # landing pad
+ .uleb128 cleanup-.LFB1
+ # no action (ie a cleanup)
+ .uleb128 0
+.LLSDACSE1:
+
+
+ .global __gcc_personality_v0
+#ifdef __PIC__
+ # Build a position independent reference to the personality function.
+ .hidden DW.ref.__gcc_personality_v0
+ .weak DW.ref.__gcc_personality_v0
+ .section .data.DW.ref.__gcc_personality_v0,"awG",@progbits,DW.ref.__gcc_personality_v0,comdat
+ .type DW.ref.__gcc_personality_v0, @object
+ .align 3
+DW.ref.__gcc_personality_v0:
+ .size DW.ref.__gcc_personality_v0, 8
+ .quad __gcc_personality_v0
+#endif
+
+ .section .note.GNU-stack,"",@progbits
+ .section .note.GNU-split-stack,"",@progbits
+ .section .note.GNU-no-split-stack,"",@progbits
diff --git a/libgcc/config/aarch64/t-stack-aarch64 b/libgcc/config/aarch64/t-stack-aarch64
new file mode 100644
index 0000000..4babb4e
--- /dev/null
+++ b/libgcc/config/aarch64/t-stack-aarch64
@@ -0,0 +1,3 @@
+# Makefile fragment to support -fsplit-stack for aarch64.
+LIB2ADD_ST += $(srcdir)/config/aarch64/morestack.S \
+ $(srcdir)/config/aarch64/morestack-c.c
diff --git a/libgcc/generic-morestack.c b/libgcc/generic-morestack.c
index b8eec4e..fe7092b 100644
--- a/libgcc/generic-morestack.c
+++ b/libgcc/generic-morestack.c
@@ -943,6 +943,7 @@ __splitstack_find (void *segment_arg, void *sp, size_t *len,
nsp -= 2 * 160;
#elif defined __s390__
nsp -= 2 * 96;
+#elif defined __aarch64__
#else
#error "unrecognized target"
#endif
--
2.1.4