Patch: Alpha32 POINTERS_EXTEND_UNSIGNED doesn't quite work right. -- take 2

[Donn Terry]

===================================================
Donn Terry                  mailto:donn@interix.com
Softway Systems, Inc.        http://www.interix.com
2850 McClelland Dr, Ste. 1800   Ft.Collins CO 80525
Tel: +1-970-204-9900           Fax: +1-970-204-9951
===================================================
Problem: Alpha32 POINTERS_EXTEND_UNSIGNED doesn't quite work right.  Take 2.

RTH submitted a fix for the above problem, which propigated
POINTERS_EXTEND_UNSIGNED into the pointer type.  That patch exposed
the latent bug below. 

This patch and RTH's patch were originally submitted together.  However,
RTH's patch may not prove applicable.  This patch remains valid,
although it may be that RTH's original patch is the only known way to cause
the bug to actually occur. 

This patch should be in place before RTH's is applied, to avoid a 
very unpretty compiler crash.

Details:
The RTH fix alone exposes a latent problem with an infinite recursion in
convert.c/c-convert.c.  (Subtitle, as suggested by a friend: lather,
rinse, repeat.)

    pointer_diff() calls convert (while setting up the MINUS).
    The operand being converted is an Alpha32 pointer (signed SI).
    (Other calls trigger the same problem.)

    Lather:
    Convert checks for various no-ops and then calls convert_to_integer.
    Convert to integer determines that it's converting from a
    pointer, and does expr=fold(build1(CONVERT...).

    That call to fold does nothing, and expr is returned as CONVERT.
    Convert_to_integer then calls itself with expr (now an integer
    because of the convert).

    This call to convert_to_integer determines that it has an
    integer and after checking that the current operation is not
    one of a few special cases determines if it is narrowing or
    widening the type.  If it's widening it (or it's a no-size-change)
    it converts this to a NOP_EXPR (so now we have NOP_EXPR(CONVERT(ptr...  ).

    Here we return to the outer convert_to_integer() which returns
    immediately to the original call in convert which is
    fold(convert_to_integer....

    Rinse:

    fold determines that this is NOP_EXPR(CONVERT(.... and treats
    that as a special case, see "Handle cases of two conversions
    in a row".  This ends up taking the branch where it discards
    the intermediate converison, Resulting in the object

    contained two levels down in the NOP_EXPR, which our original
    argument to convert, unmodified. 

    Repeat:

    convert() is then called on the resulting (unmodified) object.
    At this point, we recurse.

To break the recursion, we note that the "Handle cases..." code can
have the effect of undoing a conversion from a pointer to an integer
or vice-versa (because it treats pointers and integers equivalently).
If it is going to undo a conversion sequence that has a pointer on
one end and an integer on the other, don't allow that.

Fix: test for a change in type in fold().  (Note, the code there could
be hand-optimized a little, but this more clearly expresses the intent.)

Sun May 9 21:05:41 1999 Donn Terry (donn@interix.com)

	* fold-const.c (fold): Break recursion: pointer <-> integer
	conversion should not be undone by fold.
	  
diff -urP egcs.source.old/gcc/fold-const.c egcs.source/gcc/fold-const.c
--- egcs.source.old/gcc/fold-const.c	Fri Apr  9 10:02:46 1999
+++ egcs.source/gcc/fold-const.c	Mon May  3 10:18:37 1999
@@ -4550,7 +4550,8 @@ fold (expr) 
 	      && (inter_float || inter_unsignedp == inside_unsignedp)
 	      && ! (final_prec != GET_MODE_BITSIZE (TYPE_MODE (final_type))
 		    && TYPE_MODE (final_type) == TYPE_MODE (inter_type))
-	      && ! final_ptr)
+	      && ! final_ptr
+	      && final_ptr == inside_ptr)
 	    return convert (final_type, TREE_OPERAND (TREE_OPERAND (t, 0), 0));
 
 	  /* If we have a sign-extension of a zero-extended value, we can
@@ -4581,7 +4582,8 @@ fold (expr) 
 	      && ! (final_ptr && inside_prec != inter_prec)
 	      && ! (final_prec != GET_MODE_BITSIZE (TYPE_MODE (final_type))
 		    && TYPE_MODE (final_type) == TYPE_MODE (inter_type))
-	      && ! final_ptr)
+	      && ! final_ptr
+	      && final_ptr == inside_ptr)
 	    return convert (final_type, TREE_OPERAND (TREE_OPERAND (t, 0), 0));
 	}