[Patch] Overflows in array allocation.

[Ralph Loader]

Hi,

The two array allocation functions in prims.cc have overflow problems.

_Jv_NewObjectArray wasn't checking for overflow when calculating the
number of bytes for an array.

_Jv_NewPrimArray was correctly checking for overflowing a size_t, but
then truncated to jint - which is a problem on machines with 64 bit
size_t.

The following patch fixes both to ensure that the number of bytes does
not overflow a jint, i.e., ((1<<31)-1), since the lower level allocation
functions take a jint (or a jsize, which is the same thing).

A better fix might be change the allocation functions to take size_t
rather than jint / jsize.  That would change quite a few places, and I'm
not sure if / what requirements there are not to change the ABI of the
functions & objects involved.  I can do this if it's wanted.

Ralph.

2003-09-24  Ralph Loader  <suckfish@ihug.co.nz>

	* prims.cc (_Jv_NewObjectArray): Make sure byte size doesn't
	overflow a jint.
	(_Jv_NewPrimArray): Check for overflowing a jint, replacing a
	check for overflowing size_t, since the lower level functions
	take a jint.
	* testsuite/libjava.lang/newarray_overflow.java:  New file.
	* testsuite/libjava.lang/newarray_overflow.out:  New file.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: array.diff
Type: text/x-patch
Size: 3667 bytes
Desc: 
URL: <http://gcc.gnu.org/pipermail/java-patches/attachments/20030924/253f40fe/attachment.bin>