Attribute malloc is documented as: This tells the compiler that a function is malloc-like, i.e., that the pointer P returned by the function cannot alias any other pointer valid when the function returns, and moreover no pointers to valid objects occur in any storage addressed by P. The test case below shows that although GCC takes advantage of this property to eliminate impossible tests when calling __builtin_malloc it doesn't do the same when calling a user-defined function declared with the attribute. $ cat x.c && gcc -O2 -S -Wall -fdump-tree-optimized=/dev/stdout x.c void f (int **p) { int *x = *p; int **q = __builtin_malloc (sizeof (int*)); *q = 0; // *q cannot be equal to *p prior to the assignment if (x != *p) // folded to false __builtin_abort (); } __attribute__ ((malloc)) void* g (int); void h (int **p) { int *x = *p; int **q = g (sizeof (int*)); *q = 0; // *q cannot be equal to *p prior to the assignment if (x != *p) // not folded __builtin_abort (); } ;; Function f (f, funcdef_no=0, decl_uid=1906, cgraph_uid=1, symbol_order=0) f (int * * p) { <bb 2> [local count: 1073741824]: return; } ;; Function h (h, funcdef_no=1, decl_uid=1913, cgraph_uid=2, symbol_order=1) h (int * * p) { int * x; int * _1; <bb 2> [local count: 1073741824]: x_4 = *p_3(D); g (8); _1 = *p_3(D); if (_1 != x_4) goto <bb 3>; [0.00%] else goto <bb 4>; [99.96%] <bb 3> [count: 0]: __builtin_abort (); <bb 4> [local count: 1073312328]: return; }
You fail to see that g may clobber *p because p points to global memory. GCC knwos that malloc doesn't do such clobbering but you didn't tell GCC that g does not (looks like you actually can't do that right now).
My reading of the attribute malloc documentation: the pointer P returned by the function cannot alias any other pointer valid when the function returns, and moreover no pointers to valid objects occur in any storage addressed by P. is that in int **q = g (sizeof (int*)); *q = 0; // *q cannot be equal to *p prior to the assignment the assignment to *q cannot clobber any object because *q doesn't store a pointer to any storage.
This limitation is also getting in the way of detecting out-of-bounds accesses by string functions to dynamically allocated arrays, including VLAs. With the patch for pr91582 I'm testing, GCC detects the invalid access in functions f0() and f3() below, but because of this limitation, it doesn't detect the same bug in functions f1() or f2() (when the loop is unrolled). That's because __builtin_alloca_with_align is neither declared with attribute malloc nor recognized as "special" like malloc in that the pointer it returns doesn't alias any other pointer in the program. $ cat c.c && gcc -O2 -S -Wall -Wextra c.c $ cat c.c && /build/gcc-91582/gcc/xgcc -B /build/gcc-91582/gcc -O2 -S -Wall -Wextra c.c void sink (void*); void f0 (unsigned n) { if (n > 4) n = 4; char a[n]; a[4] = 0; // warning (good) sink (a); } void f1 (unsigned n) { if (n > 4) n = 4; char a[n]; a[3] = 0; a[4] = 0; // missing warning sink (a); } void f2 (unsigned n) { if (n > 4) n = 4; char a[n]; int i = 0; do { a[i] = i; // missing warning when loop is unrolled } while (i++ != 4); sink (a); } void f3 (unsigned n) { if (n > 4) n = 4; char *p = (char*)__builtin_malloc (n); int i = 0; do { p[i] = i; // warning (good) } while (i++ != 4); sink (p); } c.c: In function ‘f0’: c.c:9:8: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=] 9 | a[4] = 0; // warning (good) | ~~~~~^~~ c.c:8:8: note: at offset 4 to an object with size at most 4 declared here 8 | char a[n]; | ^ c.c: In function ‘f3’: c.c:45:10: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=] 45 | p[i] = i; // warning (good) | ~~~~~^~~ c.c:42:20: note: at offset 4 to an object with size at most 4 allocated by ‘__builtin_malloc’ here 42 | char *p = (char*)__builtin_malloc (n); | ^~~~~~~~~~~~~~~~~~~~
If (when?) this optimization is implemented, it would also be great if returning `type *restrict`, `struct somestruct { /*...*/ type *restrict p; /*...*/ }`, or an equivalent of these via a pointer (e.g., as in `void my_malloc(void *restrict*Result, size_t Sz);`) resulted in the same optimization being applied ( unless I'm mistaken in that `restrict` applied in these context implies the same (__attribute((malloc))-like) semantics).
(In reply to Petr Skocik from comment #4) PR106850 for that, pretty much.