[Bug c/51147] New: attribute((mode(byte))) on an enum generates wrong code

[pkoning at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=51147

             Bug #: 51147
           Summary: attribute((mode(byte))) on an enum generates wrong
                    code
    Classification: Unclassified
           Product: gcc
           Version: 4.5.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c
        AssignedTo: unassigned@gcc.gnu.org
        ReportedBy: pkoning@gcc.gnu.org


Created attachment 25830
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=25830
Test case file

In an attempt to work around bug 49459, I tried putting attribute(mode_byte))
on the enum declaration (instead of on the typedef as in that bug).

While that fixes the wrong debug output, it instead gives me seriously invalid
code.  The attached testcase shows the issue.  The compiler appears to handle
the functions foo and bar as if they return 1 unconditionally, so foo is
called, bar is not, and test returns 1 unconditionally.

Looking at the tree dump files, I see in the 001t.tu file something odd about
the return type of foo() -- it is showing up as an enum (unsigned) whose min
value is 0 and max value is -1.  I'm not sure how that causes the specific
failure but it makes me wonder.  The mishandling of the function return values
shows up right away (in the 003t.original dump file).

The bug has been seen in 4.5.1 and 4.6.1.  I also tried 3.3.3 because I happen
to have it handy, but there the compiler crashes).

A 4.1.2 compiler (stock gcc on my Linux) gets it wrong also, with the addition
that it warns "Comparison is always true due to limited range of data type". 
4.5.1 does not say so (not even with -Wall).

All this is with -O2.