[Bug sanitizer/68065] Size calculations for VLAs can overflow

["msebor at gcc dot gnu.org" <gcc-bugzilla at gcc dot gnu dot org>]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68065

--- Comment #22 from Martin Sebor <msebor at gcc dot gnu.org> ---
(In reply to joseph@codesourcery.com from comment #20)
> where the C 
> standard fails to recognize limits in such areas but all implementations 
> in practice have such limits, that's a defect in the C standard).

I agree that that's one way to look at it (and the way I would prefer to see
it).  Another way is that the standard requires
sizeof(excessively-large-vla-type) to overflow/wrap.  That's how the
implementations I've tested behave, including GCC.  This could, of course, be
said to be a manifestation of undefined behavior rather than a feature.  Either
way, the result is the same and the problem with it, as was pointed out in the
WG14 discussion, is that it can lead to buffer overflow when the overflowed
size of the VLA type is used is to allocate memory on the heap and the number
of elements in the VLA to write to the memory.

> One way (for C) to get a check that is evaluated at the point where the 
> size of the type is evaluated is to hook into how grokdeclarator stores 
> array size expressions in *expr (other parts of the front end then ensure 
> that *expr is evaluated at exactly the right time).  Instead of storing 
> plain size expressions, it would be necessary to store a more complicated 
> expression containing the checks.  *And* it would be necessary to put 
> checks in there for the case where a constant-size array is declared but 
> the elements of that array are of variable size, to make sure that 
> overflow in the multiplication is detected in that case (right now, *expr 
> is used only for variable size expressions, because they might have side 
> effects so the time of evaluation matters).  This applies regardless of 
> what the conditions are for when to enable such a check.

Yes.  This is close to what C++ new expressions already do, except that they
throw an exception rather than trapping (this is required by the C++ standard).