[Bug c/13182] New: -fstack-check probes too distant when allocating on stack

[schneck at math dot berkeley dot edu]

The stack probes generated by -fstack-check appear to be too distant
when something gets allocated on the stack.

Running "gcc -fstack-check -S tmp.c" where tmp.c is
<code>
f () {
  int a;
  int foo[2100];
}
int g() {
  int a;
  int foo[2100];
  f ();
}
</code>
produces the following output in tmp.s:
<code>
        .file   "tmp.c"
        .text
.globl _f
        .def    _f;     .scl    2;      .type   32;     .endef
_f:
        pushl   %ebp
        movl    %esp, %ebp
        subl    $24, %esp
        movl    %esp, -12(%ebp)
        leal    -12792(%esp), %edx
        movl    $0, (%edx)
        movl    $8400, -8(%ebp)
        movl    -8(%ebp), %eax
        call    __alloca
        movl    -12(%ebp), %esp
        leave
        ret
.globl _g
        .def    _g;     .scl    2;      .type   32;     .endef
_g:
        pushl   %ebp
        movl    %esp, %ebp
        pushl   %ebx
        subl    $20, %esp
        leal    -4392(%esp), %eax
        movl    $0, (%eax)
        movl    %esp, %ebx
        leal    -12792(%esp), %eax
        movl    $0, (%eax)
        movl    $8400, -12(%ebp)
        movl    -12(%ebp), %eax
        call    __alloca
        call    _f
        movl    %ebx, %esp
        movl    -4(%ebp), %ebx
        leave
        ret
</code>

In f the first probe is at -12792, skipping over three whole pages!
In g, a non-leaf function, at least we first probe -4392, but there
are still two pages skipped between the two probes.

I think the culprit in the code (gcc-3.3.1-3 on cygwin)
is in the probe_stack_range function in explow.c:
<code>
      /* Start probing at FIRST + N * STACK_CHECK_PROBE_INTERVAL
         for values of N from 1 until it exceeds LAST.  If only one
         probe is needed, this will not generate any code.  Then probe
         at LAST.  */
      for (offset = first + STACK_CHECK_PROBE_INTERVAL;
           offset < INTVAL (size);
           offset = offset + STACK_CHECK_PROBE_INTERVAL)
        emit_stack_probe (gen_rtx_fmt_ee (STACK_GROW_OP, Pmode,
                                          stack_pointer_rtx,
                                          GEN_INT (offset)));

      emit_stack_probe (gen_rtx_fmt_ee (STACK_GROW_OP, Pmode,
                                        stack_pointer_rtx,
                                        plus_constant (size, first)));
</code>
It looks like there is a confusion about what 
probe_stack_range(first,size) means... is it probe from 
sp down to sp-size assuming that sp to sp-first is already probed?
Or is it to probe down to sp-(first+size)?

Additionally, it looks like it's being assumed that 4392 has already 
been probed, even in leaf functions where that has not been done.

Finally, can anyone tell me what the invariant guaranteed by 
-fstack-check is?  I think that it is "At entry to any function,
sp-4392 is guaranteed to be above the guard page."  (Then the probe in 
non-leaf functions is to preserve the guarantee in any functions that 
are called.)  Is that correct?

Robert

-- 
           Summary: -fstack-check probes too distant when allocating on
                    stack
           Product: gcc
           Version: 3.3.1
            Status: UNCONFIRMED
          Severity: minor
          Priority: P3
         Component: c
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: schneck at math dot berkeley dot edu
                CC: gcc-bugs at gcc dot gnu dot org
 GCC build triplet: i686-pc-cygwin
  GCC host triplet: i686-pc-cygwin
GCC target triplet: i686-pc-cygwin


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=13182