This is the mail archive of the gcc-bugs@gcc.gnu.org mailing list for the GCC project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

[Bug c/80710] New: Stack smashing detected in correct code depending on optimization flag


https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80710

            Bug ID: 80710
           Summary: Stack smashing detected in correct code depending on
                    optimization flag
           Product: gcc
           Version: unknown
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: c
          Assignee: unassigned at gcc dot gnu.org
          Reporter: dr.markus.hoffmann at gmx dot de
  Target Milestone: ---

Stack smashing detected if the code is compiled with -O1 or with
-fomit-frame-pointer. Everything fine, when compiled without optimization or
with -fno-omit-frame-pointer

Example code follows:
[code]

#include <stdlib.h>
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <string.h>

typedef struct {
  int a;
  int b;
  int c;
  int d;
} A;


A test2(int n) {
  A ret;
  printf("Hello n=%d\n",n);
  ret.a=1;
  ret.b=2;
  ret.c=3;
  return(ret);
}

#define GTT_SIZE 8

typedef struct  {long feld[GTT_SIZE];} GTT;
long (*adr)(GTT);


void dummy() {
  long ret;
  GTT gtt;
  A t;
  int i;

  printf("adr t: %p\n",&t);
  for(i=0;i<GTT_SIZE;i++) gtt.feld[i]=i;
  gtt.feld[0]=(long)&t;
  gtt.feld[1]=5;

  adr=(long (*)(GTT))test2;
  ret=adr(gtt);
  printf("Function returned: 0x%x\n",(unsigned int)ret);
  if(ret==(long)&t) printf("This is adress of t\n");
  printf("Function ret: %d %d %d\n",t.a,t.b,t.c);

  printf("original stack: \n");
  for(i=0;i<GTT_SIZE;i++) {
    printf("%d : $%x\n",i,(unsigned int)gtt.feld[i]);
  }
// >>>>>>>The ERROR is triggered here !!!!!
}

int main() {
  printf("This is main.\n");
  dummy();
// <<<<<<< this will never be reached.
  printf("End of main.\n");
}

[/code]

Output: (normal case 
[code]
This is main.
adr t: 0xbfd76d7c
Hallo n=5
Function returned: 0xbfd76d7c
This is adress of t
Function ret: 1 2 3
original stack: 
0 : $bfd76d7c
1 : $5
2 : $2
3 : $3
4 : $4
5 : $5
6 : $6
7 : $7
End of main.

[/code]

Output if comiled with "gcc -fomit-frame-pointer a.c"
[code]
This is main.
adr t: 0xbfa1f30c
Hallo n=5
Function returned: 0xbfa1f30c
This is adress of t
Function ret: 2 3 0
original stack: 
0 : $5
1 : $2
2 : $3
3 : $4
4 : $5
5 : $6
6 : $7
7 : $9fc2c100
*** stack smashing detected ***: ./a.out terminated
Abort
[/code]

The latter should not happen and is considered as a bug in gcc.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]