[PATCH] Fix fold_widened_comparison (PR tree-optimization/19283)

[Jakub Jelinek]

Hi!

This testcase is miscompiled because fold_widened_comparison
"optimizes"
unsigned int u; /* global var */
(short int) (short unsigned int) (int) (short unsigned int) u < 0
into 1.
The problem is that fold_widened_comparison expects get_unwidened
to return something with a shorter type than the original, unless
it returns the passed argument untouched.
But get_unwidened doesn't guarantee that, it can return something
with the same precision, but can change signedness of the type, as
in this case it returns (short unsigned int) u.
get_unwidened's comment IMHO document that:
If we have not stripped any zero-extensions (uns is 0), we can strip any
kind of extension.).
But arg1 in this case is (short int) 0 and arg1_unw as well,
so when computing max as (short int) ~ (unsigned short int) 0
we get -1 with TREE_OVERFLOW set and because of the overflow do the
wrong thing.

I have looked at fold_widened_comparison and in both optimizations
it does it IMHO relies on shorter_type being actually shorter,
the "do comparison in shorter mode" optimization when the mode is
not shorter is useless and there is fold_sign_changed_comparison
called right after fold_widened_comparison that handles the cases
that are useful.
The "does not fit into the range of the shorter type" optimization
relies on the type being shorter.

2005-01-06  Jakub Jelinek  <jakub@redhat.com>

	PR tree-optimization/19283
	* fold-const.c (fold_widened_comparison): Return NULL if shorter_type
	is not shorter than the original type.

	* gcc.c-torture/execute/20050106-1.c: New test.

--- gcc/fold-const.c.jj	2005-01-06 14:33:12.000000000 +0100
+++ gcc/fold-const.c	2005-01-06 17:23:39.092995159 +0100
@@ -5993,7 +5993,10 @@ fold_widened_comparison (enum tree_code 
   if (arg0_unw == arg0)
     return NULL_TREE;
   shorter_type = TREE_TYPE (arg0_unw);
-  
+
+  if (TYPE_PRECISION (TREE_TYPE (arg0)) <= TYPE_PRECISION (shorter_type))
+    return NULL_TREE;
+
   arg1_unw = get_unwidened (arg1, shorter_type);
   if (!arg1_unw)
     return NULL_TREE;
--- gcc/testsuite/gcc.c-torture/execute/20050106-1.c.jj	2005-01-06 17:26:46.273803517 +0100
+++ gcc/testsuite/gcc.c-torture/execute/20050106-1.c	2005-01-06 17:25:22.000000000 +0100
@@ -0,0 +1,19 @@
+/* PR tree-optimization/19283 */
+
+void abort (void);
+
+static inline unsigned short
+foo (unsigned int *p)
+{
+  return *p;
+};
+
+unsigned int u;
+
+int
+main ()
+{
+  if ((foo (&u) & 0x8000) != 0)
+    abort ();
+  return 0;
+}

	Jakub