[Bug c++/11667] New: wider-than-ine enums never compare equal to 0

["aoliva at gcc dot gnu dot org" <gcc-bugzilla at gcc dot gnu dot org>]

PLEASE REPLY TO gcc-bugzilla@gcc.gnu.org ONLY, *NOT* gcc-bugs@gcc.gnu.org.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=11667

           Summary: wider-than-ine enums never compare equal to 0
           Product: gcc
           Version: 3.4
            Status: UNCONFIRMED
          Severity: normal
          Priority: P2
         Component: c++
        AssignedTo: mark at codesourcery dot com
        ReportedBy: aoliva at gcc dot gnu dot org
                CC: gcc-bugs at gcc dot gnu dot org
  GCC host triplet: an

Mark,

Your patch for PR 10796, that implemented C++ DR377, also contained a
code clean-up that was actually wrong.  The underlying type of an enum
must be an integral type, but, after your patch, we end up with a type
whose precision is given by the min and max values of the enumeration,
that doesn't necessarily match the precision of any of the integral
types.

Richard Sandiford found out we were miscompiling libstdc++ because of
this, and I turned the code snippet he extracted from libstdc++ into
the attached testcase.  Richard also analyzed the problem in detail,
but it turned out that he didn't know for sure what had been decided
for DR377, so I ended up taking the issue over and found out the exact
bits of Mark's patch that introduced the problem but didn't contribute
to the implementation of the DR or the fix to the PR.

<quote author="Richard Sandiford">
When compiling "gx == 0", we call shorten_compare with:

    op0 = <nop_expr type <integer_type SI> arg 0 <var_decl gx>>
    op1 = <integer_cst type <int> constant 0>
    result_type = <integer_type SI>

The first thing shorten_compare does is to remove integer promotions, so:

    primop0 = <var_decl gx>
    primop1 = <integer_cst type <int> constant 0>

We then have:

  /* If comparing an integer against a constant more bits wide,
     maybe we can deduce a value of 1 or 0 independent of the data.
     Or else truncate the constant now
     rather than extend the variable at run time.

     This is only interesting if the constant is the wider arg.
     Also, it is not safe if the constant is unsigned and the
     variable arg is signed, since in this case the variable
     would be sign-extended and then regarded as unsigned.
     Our technique fails in this case because the lowest/highest
     possible unsigned results don't follow naturally from the
     lowest/highest possible values of the variable operand.
     For just EQ_EXPR and NE_EXPR there is another technique that
     could be used: see if the constant can be faithfully represented
     in the other operand's type, by truncating it and reextending it
     and see if that preserves the constant's value.  */

  if (!real1 && !real2
      && TREE_CODE (primop1) == INTEGER_CST
      && TYPE_PRECISION (TREE_TYPE (primop0)) < TYPE_PRECISION (*restype_ptr))
    {
      int min_gt, max_gt, min_lt, max_lt;
      tree maxval, minval;
      /* 1 if comparison is nominally unsigned.  */
      int unsignedp = TREE_UNSIGNED (*restype_ptr);
      tree val;

      type = c_common_signed_or_unsigned_type (unsignedp0,
					       TREE_TYPE (primop0));

      /* If TYPE is an enumeration, then we need to get its min/max
	 values from it's underlying integral type, not the enumerated
	 type itself.  */
      if (TREE_CODE (type) == ENUMERAL_TYPE)
--->	type = c_common_type_for_size (TYPE_PRECISION (type), unsignedp0);

Before Mark's patch, TYPE_PRECISION (TREE_TYPE (gx)) was 32, so this
code wasn't executed.  But now gx's precision is 17.

At the indicated line, "type" becomes the unsigned version of the
underlying type (i.e. an unsigned SI type).  We then convert the
minimum and maximum values of this type into the "common type",
which is signed in this case:

      maxval = TYPE_MAX_VALUE (type);
      minval = TYPE_MIN_VALUE (type);
      [...]
      if (type != *restype_ptr)
	{
	  minval = convert (*restype_ptr, minval);
	  maxval = convert (*restype_ptr, maxval);
	}

maxval is now -1 and any non-negative number seems out of range.
</quote>