This is the mail archive of the
gcc-patches@gcc.gnu.org
mailing list for the GCC project.
loc_mentioned_in_p invokes undefined behavior
- From: Alexandre Oliva <aoliva at redhat dot com>
- To: gcc-patches at gcc dot gnu dot org
- Date: Mon, 05 Nov 2007 05:27:38 -0200
- Subject: loc_mentioned_in_p invokes undefined behavior
I'd noticed that loc_mentioned_in_p() wouldn't find loc if it was
mentioned as one of the XVECEXPs of say a concatn, a parallel or
somesuch. Debugging this, I realized this function would access every
fld of an RTX as if it was a pointer to an RTX itself. This is wrong:
it makes room for false positives and it accesses as pointers fields
that might have been initialized as narrower or otherwise non-pointer
values, thus invoking undefined behavior.
This patch, that I've already tested in the vta branch, and I'm not
re-testing in mainline (x86_64-linux-gnu for both), should fix this
problem. Ok to install?
for gcc/ChangeLog .vta?
from Alexandre Oliva <aoliva@redhat.com>
* rtlanal.c (loc_mentioned_in_p): Test XVECEXPs correctly.
Index: gcc/rtlanal.c
===================================================================
--- gcc/rtlanal.c.orig 2007-11-05 04:26:09.000000000 -0200
+++ gcc/rtlanal.c 2007-11-05 04:27:22.000000000 -0200
@@ -2989,16 +2989,15 @@ loc_mentioned_in_p (rtx *loc, const_rtx
fmt = GET_RTX_FORMAT (code);
for (i = GET_RTX_LENGTH (code) - 1; i >= 0; i--)
{
- if (loc == &in->u.fld[i].rt_rtx)
- return 1;
if (fmt[i] == 'e')
{
- if (loc_mentioned_in_p (loc, XEXP (in, i)))
+ if (loc == &XEXP (in, i) || loc_mentioned_in_p (loc, XEXP (in, i)))
return 1;
}
else if (fmt[i] == 'E')
for (j = XVECLEN (in, i) - 1; j >= 0; j--)
- if (loc_mentioned_in_p (loc, XVECEXP (in, i, j)))
+ if (loc == &XVECEXP (in, i, j)
+ || loc_mentioned_in_p (loc, XVECEXP (in, i, j)))
return 1;
}
return 0;
--
Alexandre Oliva http://www.lsd.ic.unicamp.br/~oliva/
FSF Latin America Board Member http://www.fsfla.org/
Red Hat Compiler Engineer aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist oliva@{lsd.ic.unicamp.br, gnu.org}