[PATCH][RFC] Fix PR38806, wrong TBAA pruning with unions

[Richard Guenther]

On Sun, 11 Jan 2009, Richard Guenther wrote:

> 
> I believe that GCC specifies that
> 
> union U { float f; int i; };
> 
> int main()
> {
>   int i;
>   union U *u = (union U *)&i;
>   i = 1;
>   u->f = 0.0;
>   if (i != 0)
>     abort ();
>   return 0;
> }
> 
> does not abort (at least that is what comments throughout
> the GCC source suggest.  For example:
> 
>   /* If strict aliasing applies the only way to access a scalar variable
>      is through a pointer dereference or through a union (gcc extension).  */
> 
> invoke.texi specifies
> 
> "
> Similarly, access by taking the address, casting the resulting pointer
> and dereferencing the result has undefined behavior, even if the cast
> uses a union type, e.g.:
> @smallexample
> int f() @{
>   double d = 3.0;
>   return ((union a_union *) &d)->i;
> @}
> "
> 
> as a recent addition.  Does this cover the case above?  Is the only
> type-punning we allow really
> 
>   union.a = x;
>   y = union.b;
> 
> ?  In this case c_common_get_alias_set should not need to return 0
> for unions either:
> 
>   /* Permit type-punning when accessing a union, provided the access
>      is directly through the union.  For example, this code does not
>      permit taking the address of a union member and then storing
>      through it.  Even the type-punning allowed here is a GCC
>      extension, albeit a common and useful one; the C standard says
>      that such accesses have implementation-defined behavior.  */
>   for (u = t;
>        TREE_CODE (u) == COMPONENT_REF || TREE_CODE (u) == ARRAY_REF;
>        u = TREE_OPERAND (u, 0))
>     if (TREE_CODE (u) == COMPONENT_REF
>         && TREE_CODE (TREE_TYPE (TREE_OPERAND (u, 0))) == UNION_TYPE)
>       return 0;
> 
> Sigh, this exception always confuses me!

If it's really only accesses through unions that conflict the above
should return get_alias_set (TREE_OPERAND (u, 0)) instead of 0.
Thus, pretend to be an access of the whole union.  That will cause
all proper conflicts to happen (of course still also with non-union
accesses).  If we want to exclude all non-union-access conflicts
we would have to drop all alias subset recordings that do not end up
in another union (which might be sightly complicated to do).
And I wonder how much code we break there then?

Richard.