[PATCH] Reject boolean/enum types in last arg of __builtin_*_overflow_p

[Jakub Jelinek]

Hi!

As mentioned in PR71479, for __builtin_*_overflow we right now require
the last argument to be pointer to INTEGER_TYPE, not INTEGRAL_TYPE_P,
but for __builtin_*_overflow_p we were using INTEGRAL_TYPE_P.

For _Bool/bool, I'd think we could make it well defined if we wanted
(check if the infinite precision result is 0 or 1), but it is hardly useful
and as can be seen, right now we don't implement that (for bool we actually
check if the result fits into unsigned 8 bit integer, plus _Complex bool
is kind of weird), but for enums I think the definition on what is an enum
overflow is quite fuzzy, would it for C++ e.g. work differently between
-fstrict-enums and -fno-strict-enums, would it test overflows only on the
underlying type, or exact precision of the enum, something else?

So, I think it is better to continue what we've been doing already for
__builtin_*_overflow before.  We likely need to adjust the documentation
and possibly the diagnostics.  Maybe have one wording for !INTEGRAL_TYPE_P
and another wording for the != INTEGER_TYPE case (where it would complain
that the type is {pointer to,} {enum,bool,_Bool}?  Any preferences?

The patch below just uses the preexisting wording (where integral stood
for integer types including bool/enums, and integer for non-bool/enum
integer types), I'd just like to see it in soon so that people don't start
to rely on something that doesn't really work and is hard to define.

Bootstrapped/regtested on x86_64-linux and i686-linux, ok for trunk?

2016-06-10  Jakub Jelinek  <jakub@redhat.com>

	* c-common.c (check_builtin_function_arguments): Require last
	argument of BUILT_IN_*_OVERFLOW_P to have INTEGER_TYPE type.

	* c-c++-common/builtin-arith-overflow-1.c (f3): Adjust expected
	diagnostics.
	* g++.dg/ext/builtin-arith-overflow-1.C: Pass 0 instead of C
	as last argument to __builtin_add_overflow_p.

--- gcc/c-family/c-common.c.jj	2016-06-10 20:23:57.433135328 +0200
+++ gcc/c-family/c-common.c	2016-06-10 20:44:51.104848132 +0200
@@ -9999,13 +9999,19 @@ check_builtin_function_arguments (locati
       if (builtin_function_validate_nargs (loc, fndecl, nargs, 3))
 	{
 	  unsigned i;
-	  for (i = 0; i < 3; i++)
+	  for (i = 0; i < 2; i++)
 	    if (!INTEGRAL_TYPE_P (TREE_TYPE (args[i])))
 	      {
 		error_at (ARG_LOCATION (i), "argument %u in call to function "
 			  "%qE does not have integral type", i + 1, fndecl);
 		return false;
 	      }
+	  if (TREE_CODE (TREE_TYPE (args[i])) != INTEGER_TYPE)
+	    {
+	      error_at (ARG_LOCATION (i), "argument %u in call to function "
+			"%qE does not have integer type", i + 1, fndecl);
+	      return false;
+	    }
 	  return true;
 	}
       return false;
--- gcc/testsuite/c-c++-common/builtin-arith-overflow-1.c.jj	2016-06-10 20:24:01.000000000 +0200
+++ gcc/testsuite/c-c++-common/builtin-arith-overflow-1.c	2016-06-10 20:56:04.461095680 +0200
@@ -236,11 +236,11 @@ f3 (float fa, int a, _Complex long int c
   x += __builtin_sub_overflow_p (ca, b, eb);	/* { dg-error "argument 1 in call to function\[^\n\r]*does not have integral type" } */
   x += __builtin_mul_overflow_p (a, fb, bb);	/* { dg-error "argument 2 in call to function\[^\n\r]*does not have integral type" } */
   x += __builtin_add_overflow_p (a, pb, a);	/* { dg-error "argument 2 in call to function\[^\n\r]*does not have integral type" } */
-  x += __builtin_sub_overflow_p (a, eb, eb);
-  x += __builtin_mul_overflow_p (a, bb, bb);
-  x += __builtin_add_overflow_p (a, b, fa);	/* { dg-error "argument 3 in call to function\[^\n\r]*does not have integral type" } */
-  x += __builtin_sub_overflow_p (a, b, ca);	/* { dg-error "argument 3 in call to function\[^\n\r]*does not have integral type" } */
-  x += __builtin_mul_overflow_p (a, b, c);	/* { dg-error "argument 3 in call to function\[^\n\r]*does not have integral type" } */
+  x += __builtin_sub_overflow_p (a, eb, eb);	/* { dg-error "argument 3 in call to function\[^\n\r]*does not have integer type" } */
+  x += __builtin_mul_overflow_p (a, bb, bb);	/* { dg-error "argument 3 in call to function\[^\n\r]*does not have integer type" } */
+  x += __builtin_add_overflow_p (a, b, fa);	/* { dg-error "argument 3 in call to function\[^\n\r]*does not have integer type" } */
+  x += __builtin_sub_overflow_p (a, b, ca);	/* { dg-error "argument 3 in call to function\[^\n\r]*does not have integer type" } */
+  x += __builtin_mul_overflow_p (a, b, c);	/* { dg-error "argument 3 in call to function\[^\n\r]*does not have integer type" } */
   return x;
 }
 
--- gcc/testsuite/g++.dg/ext/builtin-arith-overflow-1.C.jj	2016-06-08 21:01:25.000000000 +0200
+++ gcc/testsuite/g++.dg/ext/builtin-arith-overflow-1.C	2016-06-10 20:55:04.836870794 +0200
@@ -1,11 +1,11 @@
 // { dg-do compile }
 
-enum A { B = 1, C = 2, D = __builtin_add_overflow_p (B, C, C) };
-int e[__builtin_add_overflow_p (B, C, C) + 1];
+enum A { B = 1, C = 2, D = __builtin_add_overflow_p (B, C, 0) };
+int e[__builtin_add_overflow_p (B, C, 0) + 1];
 template <int N> int foo (int);
 
 void
 bar ()
 {
-  foo <__builtin_add_overflow_p (B, C, C) + 1> (0);
+  foo <__builtin_add_overflow_p (B, C, 0) + 1> (0);
 }

	Jakub