[Bug c/16412] New: Missing 32- to 64-bit sign extensions when doing pointer arithmetics.

["drab at kepler dot fjfi dot cvut dot cz" <gcc-bugzilla at gcc dot gnu dot org>]

The following program:
/* -- y.c -------------------------------------------- */
static void foo(char *buf, unsigned int a, int b, unsigned int c)
{
        int e = a + b;
        char *ptr;

        e--;
        ptr = buf + e * c;
        ptr[0]=0;
}

int main ()
{
        char buf[4096];

        foo (&buf[1500], 0, -1, 1);
        return 0;
}
/* --------------------------------------------------- */
causes a segmentation fault when compiled with
gcc version 3.5.0 20040706 (experimental)
and run. It probably applies to older compilers as well (also tested with
gcc version 3.3.2 (Mandrake Linux 10.0 3.3.2-6.1mdk) and
gcc version 3.4.0 (Mandrake Linux 10.0 3.4.0-0.1mdk) with exactly the same results).

I tried following compiling commands (assuming the above code is in file y.c):

gcc -O0 -o y y.c
gcc -O1 -o y y.c
gcc -O2 -o y y.c
gcc -O3 -o y y.c

The first two triggers the error. The last two does not, but only because they
optimize the whole code down to almost the equivalent of 'int main() { return 0;
}'. But for instatnce the following command lines trigger the error as well:

gcc -O2 -fforce-addr -o y y.c
gcc -O3 -fforce-addr -o y y.c

>From my analysis of the problem it is caused by the fact that for example the
last two compile commands (given above) effectively reduces the program down to
the following 5 assembler instructions:

main:
       subq    $3984, %rsp
       movl    $4294967294, %eax
       movb    $0, 1380(%rsp,%rax)
       xorl    %eax, %eax
       addq    $3984, %rsp
       ret

While in fact it should be something like this (which actually works):

main:
       subq    $3984, %rsp
       mov     $-2, %rax
       movb    $0, 1380(%rsp,%rax)
       xorl    %eax, %eax
       addq    $3984, %rsp
       ret

Because otherwise the offset $-2 represented by the number $4294967294
in the first case (it is a 32-bit representation of -2) is not sign-extended to
the full 64-bit %rax register and though taken as +4294967294 instead of the
expected -2. That causes the segmentation fault.

The problem may be in that the multiplication of 'int' and 'unsigned int' is
treated as 'unsigned int' and not 'int' (I hope it's not intentional, it
wouldn't make sense).

In case it is needed, here is the report of my 'gcc -v' command:

$ gcc -v
Reading specs from /usr/local/lib64/gcc/x86_64-pc-linux-gnu/3.5.0/specs
Configured with: ../../../gcc-CVS-20040707/gcc-CVS-20040707/configure
--host=x86_64-pc-linux-gnu --prefix=/usr/local --sysconfdir=/etc
--libdir=/usr/local/lib64 --libexecdir=/usr/local/lib64 --sharedstatedir=/var
--localstatedir=/var --infodir=/usr/local/share/info
--mandir=/usr/local/share/man --with-x-includes=/usr/X11R6/include
--with-x-libraries=/usr/X11R6/lib64 --enable-shared --enable-static
--with-gnu-as --with-gnu-ld --enable-threads
--enable-version-specific-runtime-libs --with-system-zlib --disable-checking
--enable-multilib --with-x --enable-fast-character --enable-hash-synchronization
--enable-gather-detailed-mem-stats --with-libbanshee
--enable-languages=c,c++,f95,objc --with-arch=k8
Thread model: posix
gcc version 3.5.0 20040706 (experimental)

-- 
           Summary: Missing 32- to 64-bit sign extensions when doing pointer
                    arithmetics.
           Product: gcc
           Version: 3.5.0
            Status: UNCONFIRMED
          Severity: critical
          Priority: P1
         Component: c
        AssignedTo: unassigned at gcc dot gnu dot org
        ReportedBy: drab at kepler dot fjfi dot cvut dot cz
                CC: gcc-bugs at gcc dot gnu dot org
 GCC build triplet: x86_64-pc-linux-gnu
  GCC host triplet: x86_64-pc-linux-gnu
GCC target triplet: x86_64-pc-linux-gnu


http://gcc.gnu.org/bugzilla/show_bug.cgi?id=16412