[Bug libitm/93782] New: ICE (segfault) while using TM in conjunction with UBSAN

[stefansf at linux dot ibm.com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=93782

            Bug ID: 93782
           Summary: ICE (segfault) while using TM in conjunction with
                    UBSAN
           Product: gcc
           Version: 9.2.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: libitm
          Assignee: unassigned at gcc dot gnu.org
          Reporter: stefansf at linux dot ibm.com
  Target Milestone: ---

Consider the following example:

void foo(int n) {
    __transaction_atomic {
        char a[8];
        a[n] = 42;
    }
}

Using GCC release 9.2.1 or abe13e1847fb, compiling the example results in a
segfault:

gcc -fsanitize=undefined -fgnu-tm -c test.c                                     
during GIMPLE pass: *diagnose_tm_blocks                                         
test.c: In function 'foo':
test.c:1:6: internal compiler error: Segmentation fault
    1 | void foo(int n) {
      |      ^~~                                          
0x21644bb crash_signal
        /gcc/gcc/toplev.c:328
0x2173390 diagnose_tm_1
        /gcc/trans-mem.c:650
...

UBSAN introduces several function calls to internal functions which do not have
a tree representation. However, in file `trans-mem.c` line 637 such a
representation is queried `tree fn = gimple_call_fn (stmt);` resulting in a
null pointer which gets unconditionally dereferenced. This also happens at
other places as for example in function `ipa_tm_scan_irr_block`.

The GIMPLE pass `diagnose_tm_blocks` is run before the internal functions,
which are introduced by UBSAN, are expanded which happens in `pass_ubsan`.