optimization/7001: gcc-3.1 and 3.04 on x86 generate bad memset code

[adam at yggdrasil dot com]

>Number:         7001
>Category:       optimization
>Synopsis:       gcc-3.1 and 3.04 on x86 generate bad memset code
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    unassigned
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Jun 12 00:36:01 PDT 2002
>Closed-Date:
>Last-Modified:
>Originator:     adam@yggdrasil.com
>Release:        gcc-3.1 and gcc-3.0.4
>Organization:
>Environment:
Linux-2.5.19, glibc-2.2.5
>Description:
gcc-3.1 and gcc-3.0.4 optimizers under -O1 or -O2
miscompile a memset (bzero really) call in at line 607 of
XFree86-4.2/lib/Xt/Composite.c (in GetResources).
That bad memset correctly clears the memory in blocks of
four bytes, but botches the test for whether it has to
clear the last 0-3 bytes.  Instead of checking the last
two bits of the byte count, it checks the last two bits of
the destination addresss.

Although the manifestation of this bug is that the last 0-3
X resources in a toolkit application may be set to garbage
values because their corresponding "found" flags were not
initially cleared, this compiler bug probably is occurring
in lots of system software.  Just about every program uses
memset.  (I tried to submit this bug as high priority, but
your bug reporting configuration prohibits it for bug
reports from the unwashed masses.)

Here is the miscompiled line 607 source code from
memset.c, bracketed with __asm__ comments that I added
to show the miscompiled assembler code.

    /* Mark each resource as not found on arg list */
    __asm__("# BEFORE BUG");
    bzero((char *) found, (int) (num_resources * sizeof(Boolean)));
    __asm__("# AFTER BUG");
    bzero((char *) typed, (int) (num_resources * sizeof(int)));

Here is the relevant miscompiled assembler code,
after I commented out the two wrong instructions and
replaced them with correct ones:

        .loc 1 93 0
#NO_APP
        leal    -824(%ebp), %edi
        movl    $0, %edx
        cld
        movl    28(%ebp), %ecx
        shrl    $2, %ecx
        movl    %edx, %eax
        rep
        stosl
        # movl  %edi, %eax              # AJR commented out bad instruction
        movl    28(%ebp), %eax          # AJR added good instruction
        andl    $2, %eax
        testl   %eax, %eax
        je      .L13
        movw    $0, (%edi)
        addl    $2, %edi
.L13:
        # movl  %edi, %eax              # AJR commented out bad instruction
        movl    28(%ebp), %eax          # AJR added good instruction
        andl    $1, %eax
        testl   %eax, %eax
        je      .L14
        movb    %dl, (%edi)
.L14:
        .loc 1 94 0
#APP



	A final bizarre note: gcc-3.1 seems to be generating
different output from the same input file under different
runs.  For example, I just recompiled the same .i file
(i.e., the same cpp output) from this example, and this
time it generated memcpy code that simply assumed that
the byte count was a multiple of four and did not generate
any code to deal with the last 3 bytes.  This suggests
to me that wherever gcc is generating its own memset
code, it is relying on an unitialized value that is
supposed to convey some things that it knows about
the byte count.
>How-To-Repeat:
On my system, I can see the bug by running xterm or
netscape.  xterm opens as an icon instead of a regular
window because one of the last three resources in one of
those resource lists is the "iconic" option for the
top level window.

>Fix:
I don't know yet.
>Release-Note:
>Audit-Trail:
>Unformatted: