A weird crash

[Zack Weinberg]

On Fri, Jan 21, 2000 at 04:38:47AM -0000, D. J. Bernstein wrote:
> gcc 2.95.2 -O2 on a Pentium makes the following program crash:
> 
>    void big(long long u) { }
>    void doit(unsigned int a,unsigned int b,char *id)
>    { big(*id); big(a); big(b); }
>    main()
>    { doit(1,1,"\n"); }

Thank you for your bug report.  This has been corrected in the
development version.  May we add your example to the test suite?

> It's not immediately obvious to me what combination of compiler bugs
> could account for this problem. I'd like to hear a complete explanation
> of the bugs so that I can work around them.

Here is a side by side diff of the assembly output for gcc 2.95.2
(left) and the development version (right).  Please note the two
instructions I have marked with stars.

doit:                                   doit:
        pushl   %ebp                            pushl   %ebp
        movl    %esp, %ebp                      movl    %esp, %ebp
        subl    $16, %esp                       subl    $16, %esp
        pushl   %esi                            pushl   %esi
        pushl   %ebx                            pushl   %ebx
        movl    8(%ebp), %ebx                   movl    8(%ebp), %ebx
        xorl    %esi, %esi            <
        movl    12(%ebp), %eax        |         movl    12(%ebp), %esi
        xorl    %edx, %edx            *
        movl    %eax, 12(%ebp)        <
        movl    %edx, 16(%ebp)        *
        movl    16(%ebp), %eax        |         movl    16(%ebp), %ecx

        addl    $-8, %esp             |         subl    $8, %esp
        movsbl  (%eax), %edx          <         movsbl  (%ecx),%eax
        movl    %edx, %ecx            <         cltd
        sarl    $31, %ecx             <
        pushl   %ecx                  |         pushl   %edx
        pushl   %edx                  |         pushl   %eax
        call    big                             call    big

        addl    $-8, %esp             |         subl    $8, %esp
                                      <         movl    %ebx, %eax
                                      <         movl    $0, %edx
        pushl   %esi                  |         pushl   %edx
        pushl   %ebx                  |         pushl   %eax
        call    big                             call    big

        addl    $32, %esp             |         addl    $24, %esp
        addl    $-8, %esp             <
        movl    12(%ebp), %eax        |         movl    %esi, %eax
        movl    16(%ebp), %edx        |         movl    $0, %edx
        pushl   %edx                            pushl   %edx
        pushl   %eax                            pushl   %eax
        call    big                             call    big

        leal    -24(%ebp), %esp                 leal    -24(%ebp), %esp
        popl    %ebx                            popl    %ebx
        popl    %esi                            popl    %esi
        leave                                   leave
        ret                                     ret

The effect of those two instructions is to trash the pointer to string
you have passed into the function.  It is then dereferenced and the
program crashes.

Those instructions are added by global register allocation and reload,
which has changed dramatically between the two versions.  It is
unlikely that anyone can tell you exactly what is wrong or how to
avoid the bug.  My advice is don't use long long.

zw