This is the mail archive of the
gcc-patches@gcc.gnu.org
mailing list for the GCC project.
[PATCH] Harden variably_modified_type_p
- From: Richard Biener <rguenther at suse dot de>
- To: gcc-patches at gcc dot gnu dot org
- Cc: ebotcazou at adacore dot com
- Date: Thu, 17 Aug 2017 16:56:21 +0200 (CEST)
- Subject: [PATCH] Harden variably_modified_type_p
- Authentication-results: sourceware.org; auth=none
It seems with Ada one can have cycles in types via pointer types
and calling variably_modified_type_p on such type recurses indefinitely.
The following is an attempt to fix that (albeit I'm not 100% sure
there's no caller of the function using TREE_VISITED itself).
Cures c38102a.adb with LTO early debug patches for me.
Any comments? (no further testing sofar)
Richard.
Index: gcc/tree.c
===================================================================
--- gcc/tree.c (revision 251141)
+++ gcc/tree.c (working copy)
@@ -8597,8 +8601,16 @@ variably_modified_type_p (tree type, tre
case POINTER_TYPE:
case REFERENCE_TYPE:
case VECTOR_TYPE:
+ /* Ada can have pointer types refering to themselves indirectly. */
+ if (TREE_VISITED (type))
+ return false;
+ TREE_VISITED (type) = true;
if (variably_modified_type_p (TREE_TYPE (type), fn))
- return true;
+ {
+ TREE_VISITED (type) = false;
+ return true;
+ }
+ TREE_VISITED (type) = false;
break;
case FUNCTION_TYPE: