[PATCH] complex type canonicalization

[Nathan Sidwell]

This patch fixes PR 83187, a C++ ICE with alias sets.  As Jakub observed 
in the bug, we ended up with a type whose main variant's canonical 
type's main variant was not itself.  That should be an invariant.

In build_complex_type, we create a probe type:

   tree probe = make_node (COMPLEX_TYPE);
   TREE_TYPE (probe) = TYPE_MAIN_VARIANT (component_type);

notice setting TREE_TYPE to main_variant of the component type.  We then 
insert this into the canonical type hash (or find a match).

I think my type hashing clean up exposed an underlying problem, when I 
changed:
  hstate.add_object (TYPE_HASH (component_type));
to
  hashval_t hash = type_hash_canon_hash (t);

because that's abstractly better.  However, build_complex_type then 
proceeds to do two strange things.

1) regardless of whether it added a new type to the hash table, it 
checks for t == CANONICAL (t) and if equal goes to recreate the 
canonical type.

2) but it checks the canonicalness of component_type, not 
TYPE_MAIN_VARIANT (component_type), which is what the newly created 
complex's TREE_TYPE was set to.

#2 is where I think things are getting confused.  Anyway, this patch 
simply moves all of the post-insertion setting after a check to see 
whether the insertion inserted the new type, (rather than find an 
existing variant).  It also consistently uses TREE_TYPE (new_type), when 
looking at the component type in that context.

booted on x86_64-linux-gnu.  no C or C++ regressions.  ok?

nathan
-- 
Nathan Sidwell
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 83187.diff
Type: text/x-patch
Size: 5319 bytes
Desc: not available
URL: <http://gcc.gnu.org/pipermail/gcc-patches/attachments/20171128/f7499cbd/attachment.bin>