C pre-DR#7: VLA definitions

[Joseph S. Myers]

The third issue in this pre-DR is that of whether the testcases in bug 
16409 are valid or constraint violations.  The first two relate to issues 
discussed on this list, on IRC or on comp.std.c.

Pre-DR: VLA definitions
=======================

There are some problems with the C99 definitions of variable length
array types and variably modified types.

1. Variably modified types are defined by 6.7.5#3:

       [#3] A full declarator is a declarator that is not  part  of
       another  declarator.   The  end  of  a  full declarator is a
       sequence point.  If the nested sequence of declarators in  a
       full  declarator  contains a variable length array type, the
       type specified by the full declarator is said to be variably
       modified.

It is desirable for the definition to look at the declarator rather
than just the resulting type, so that function parameters adjusted
from array to pointer type are variably modified if the array size is
variable: in

    void
    f (int i, int a[static ++i])
    {
      // ...
    }

the increment of i must be evaluated for the definition of "static" in
this context to make sense.  However, what it means for the
declarators to "contain" a type is unclear.  The natural
interpretation is that they include an array declarator with array
size [*] or an expression which is not an integer constant
expression.  However, this does not cover cases such as

    int x;
    // ...
    typedef int vla[x];
    vla y[3];

where a typedef for a variably modified type is used.  y is a VLA, and
clearly ought to be variably modified, but nothing about the
declarators makes it variably modified; only the declaration specifier
does so.

2. Does "known constant size" mean something different from "not a
VLA"?  The phrase is used in the definition of composite types,
6.2.7#3:

         -- If one type is an array of  known  constant  size,  the
            composite  type is an array of that size; otherwise, if
            one type is a variable length array, the composite type
            is that type.

and in an example in 6.5.6#11 (where it doesn't cause problems), and
in 6.7.5.2#4 to define VLAs:

       [#4] If the size is  not  present,  the  array  type  is  an
       incomplete  type.   If  the  size  is  * instead of being an
       expression, the array type is a variable length  array  type
       of  unspecified size, which can only be used in declarations
       with   function   prototype   scope;122)   such  arrays  are
       nonetheless complete types.   If  the  size  is  an  integer
       constant  expression  and  the  element  type  has  a  known
       constant size, the array type is not a variable length array
       type;  otherwise,  the array type is a variable length array
       type.

Suppose the implementation does not accept any non-standard forms of
constant expressions under 6.6#10, so that (int)+1.0 is an arithmetic
constant expression but not an integer constant expression.  Thus
int[(int)+1.0] is a VLA type.  But is int[1][(int)+1.0] a VLA type?
The element type is a VLA type, but the element size is a known
constant.  If "known constant size" is interpreted to include some VLA
cases, this also means further indeterminacy of composite types in
such cases; is "an array of that size" a VLA of that size, or a
non-VLA of that size, and may cases involving compatible array types
with different known constant sizes (which would yield undefined
behavior if executed) be rejected at translation time?

3. If an incomplete array type has elements of unknown size, should
the incomplete array type be a VLA type?  The definition of VLA types
in 6.7.5.2#4 only seems to make complete types into VLA types.  In
particular, does the following, at block scope, violate any
constraint?

    int i;
    // ...
    int c[][i] = { 0 };

If it is not a VLA - and nothing in the standard seems to make it a
VLA - then the initializer would seem to be valid, and to determine
the size of the array.  This seems rather against the spirit of
prohibiting initializing VLAs in 6.5.2.5#1 (compound literals) and
6.7.8#3 (initializers).

Those appear to be the only places where it particularly matters
whether such types are VLA types.  In other cases, use of such types
does not depend on whether they are VLA types, or yields a constraint
violation whether or not they are VLA types, or in the case of

    static int c[][i];

at block scope violates the requirement of 6.7#7 for the type to be
complete, so causing undefined behavior, though if the type were a VLA
type then there would be a violation of the constraint in 6.7.5.2#2,
so requiring a diagnostic.

Defining such types to be VLA types would ensure constraint violations
in 6.5.2.5#1 and 6.7.5.2#2.  6.7.8#3 would need rewording to avoid
"that is not a variable length array type" applying to "object type"
only.

-- 
Joseph S. Myers               http://www.srcf.ucam.org/~jsm28/gcc/
  http://www.srcf.ucam.org/~jsm28/gcc/#c90status - status of C90 for GCC 4.0
    jsm@polyomino.org.uk (personal mail)
    jsm28@gcc.gnu.org (Bugzilla assignments and CCs)