misbehaviour with md5_process_bytes and maybe in optimization

Ian Lance Taylor iant@google.com
Fri Sep 23 21:45:00 GMT 2011


Pierre Vittet <piervit@pvittet.com> writes:

> Thanks for your interest,
>
> I just checked revision 179127 of GCC. Last revision is 177700, it has
> not been change for 6 weeks.
>
> My file is the same as this one:
> http://gcc.gnu.org/viewcvs/trunk/libiberty/md5.c?revision=177700&view=markup
>
> in libiberty/md5.c, function md5_process_bytes start line 203.
>
> On 23/09/2011 17:13, Ian Lance Taylor wrote:
>> Pierre Vittet <piervit@pvittet.com> writes:
>> 
>>> The bug appears when:
>>> 	1) We use libiberty compiled with -O0
>>> 	2) We first call md5_process_bytes with a less than 64 bits buffer (we
>>> call his size len1).
>>> 	3) We make a new call of md5_process_bytes with a buffer which has a
>>> size len2 such as:
>>> 	len2 > 127 + 65 (so test in line 228 of md5.C will be true)
> line 228 is the following:    if (len > 64)
>>> 	128 -len1 != Mulint with Mulint %  __alignof__ (md5_uint32) != 0 (so
>>> condition on line 238 is true)
> line 238 is the following: if (UNALIGNED_P (buffer))
>>> 	len2 - (128 - len1) = Mul64 and Mul64 such as Mul %64=0 (so the loop of
>>> line 239 is broken with len = 64, this leads to the bug as, line 249,
>>> (len & ~63) = 64 and we shift the buffer without processing the data).
>
> line 239 is the following: while (len > 64)
> line 249: buffer = (const void *) ((const char *) buffer + (len & ~63));
>> 
>> The line numbers you mention do not correspond to any version of
>> libiberty/md5.c that I can see.  Can you list the exact line for each
>> line number you mention, so that your explanation is easier to follow?
>> Thanks.
>
> I give about the same explanation in the README (which is in the
> attached archive of my previous mail) but I does not use line number but
> direct quote of the code. It mights be more easy to try the plugin with
> gdb but it needs to compile libiberty.a with -O0.

Thanks, I think I have it sorted out now.

It does not happen on x86 glibc-based systems at -O2 because at -O2
<string.h> #defines STRING_ARCH_unaligned, so the problematic code is
not compiled or executed.

The error was introduced by this change:

2005-07-03  Steve Ellcey  <sje@cup.hp.com>

	PR other/13906
	* md5.c (md5_process_bytes): Check alignment.

Thanks for noticing this problem, analyzing it, and reporting it.

I committed this patch to mainline to fix the problem.  Bootstrapped on
x86_64-unknown-linux-gnu.

Ian


2011-09-23  Ian Lance Taylor  <iant@google.com>

	* md5.c (md5_process_bytes): Correct handling of unaligned
	buffer.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: foo.patch
Type: text/x-diff
Size: 951 bytes
Desc: patch
URL: <http://gcc.gnu.org/pipermail/gcc-patches/attachments/20110923/bc6e9462/attachment.bin>


More information about the Gcc-patches mailing list