c/10260: alloca 0xFFFFFFFF and -fstack-check

[Kevin Ryde]

>Number:         10260
>Category:       c
>Synopsis:       alloca 0xFFFFFFFF and -fstack-check
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    unassigned
>State:          open
>Class:          wrong-code
>Submitter-Id:   net
>Arrival-Date:   Sat Mar 29 00:36:00 UTC 2003
>Closed-Date:
>Last-Modified:
>Originator:     
>Release:        3.2.1 20020830 (Debian prerelease) (Debian testing/unstable)
>Organization:
>Environment:
System: Linux blah 2.2.15 #1 Tue Apr 25 17:13:48 EST 2000 i586 unknown
Architecture: i586
	<machine, os, target, libraries (multiple lines)>
host: i386-pc-linux-gnu
build: i386-pc-linux-gnu
target: i386-pc-linux-gnu
configured with: /mnt/data/gcc-3.1/gcc-3.2-3.2.1ds0/src/configure -v --enable-languages=c,c++,java,f77,proto,objc,ada --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-gxx-include-dir=/usr/include/c++/3.2 --enable-shared --with-system-zlib --enable-nls --without-included-gettext --enable-__cxa_atexit --enable-java-gc=boehm --enable-objc-gc i386-linux

>Description:
When a program attempts to alloca 0xFFFFFFFF bytes under
-fstack-check, a stack overflow exception is not provoked.

>How-To-Repeat:
The program foo.c below compiled and run

	gcc -fstack-check foo.c
	./a.out

successfully reaches and executes the printf, whereas I had hoped it
would get a segv, which is what happens when a smaller alloca amount
like 0xFFFF0000 is requested.


Looking at the code generated I guess a size like 0xFFFFFFFF is
rounded up to a multiple of the stack alignment, giving 0.  And the
4392 byte safety margin makes sizes like 0xFFFFFF00 wrap around to
smallish positive values too.

In normal circumstances I wouldn't think this is a problem, since
anyone asking for such sizes deserves everything they get, but when
-fstack-check is in use I think it'd be highly desirable to ensure a
system stack overflow is provoked for any size bigger than the system
can handle.


The same seems to be true of -fstack-limit-symbol (in the current gcc
cvs), but only in respect of the rounding up, so 0xFFFFFFFF is not
detected as an overflow, but sizes up to 0xFFFFFFF0 are.



--=-=-=
Content-Type: text/x-csrc
Content-Disposition: attachment; filename=foo.c

volatile unsigned n = 0xFFFFFFFF;

int
main (void)
{
  char *p;
  p = __builtin_alloca (n);
  printf ("%p\n", p);
}

--=-=-=--
>Fix:
>Release-Note:
>Audit-Trail:
>Unformatted:
 --=-=-=