Re: [patch i386] Enable attribute based calling convention switching and support vaarg.

[Bob Wilson <bwilson at tensilica dot com>]

Kai Tietz wrote:
> I found the problem. It is related to the fact, that the native
> va_list type for x86_64 is a structure. In earlier patches the type
> was generated twice by the helper function
> x86_build_builtin_va_list_abi. This lead to the problem, that there
> are two record types use, which brings the gimple to fail.
> Additionally the canonical_va_list_type method needs to handle
> va_list_type_node, too. Additionally it has to dereference pointers to
> va_list types.

This still breaks Xtensa, which also has a struct va_list.  The 
std_canonical_va_list_type function returns NULL_TREE when called from 
expand_builtin_va_start through stabilize_va_list, so that stabilize_va_list 
asserts.  It correctly returns va_list_type_node when called from 
gimplify_va_arg_expr.  I think the difference is that in the former case 
std_canonical_va_list_type is given a pointer to the struct type, and it does 
not "dereference" the pointer type to get to the struct:

  if (INDIRECT_REF_P (type))
    type = TREE_TYPE (type);
  else if (POINTER_TYPE_P (type) && POINTER_TYPE_P (TREE_TYPE(type)))
    type = TREE_TYPE (type);

It can handle pointers to pointers, but not pointers to structs.  I'm not 
familiar with this code and can't easily suggest a fix.

In the short term, I can work around this by defining 
TARGET_CANONICAL_VA_LIST_TYPE for Xtensa, but that doesn't seem like the right 
thing to do.