Confirmed, there's reduced test-case for C:

typedef __int20 unsigned a;
int b, c (c > a (-1) / b);

./xgcc -B. tc.ii
tc.ii: In function ‘void __static_initialization_and_destruction_0(int, int)’:
tc.ii:2:13: internal compiler error: in simplify_subreg, at simplify-rtx.c:6034
 int b, c (c > a (-1) / b);
           ~~^~~~~~~~~~~~
0x105d288 simplify_subreg(machine_mode, rtx_def*, machine_mode, unsigned int)
	../../gcc/simplify-rtx.c:6034
0x105dfd7 simplify_gen_subreg(machine_mode, rtx_def*, machine_mode, unsigned int)
	../../gcc/simplify-rtx.c:6265
0x105e07e lowpart_subreg(machine_mode, rtx_def*, machine_mode)
	../../gcc/simplify-rtx.c:6288
0xc40370 gen_lowpart_common(machine_mode, rtx_def*)
	../../gcc/emit-rtl.c:1456
0x103c314 gen_lowpart_general(machine_mode, rtx_def*)
	../../gcc/rtlhooks.c:47
0xc7816d convert_move(rtx_def*, rtx_def*, int)
	../../gcc/expr.c:530
0xc77f96 convert_move(rtx_def*, rtx_def*, int)
	../../gcc/expr.c:508
0xc78ad5 convert_modes(machine_mode, machine_mode, rtx_def*, int)
	../../gcc/expr.c:685
0xdf2639 expand_arith_overflow_result_store
	../../gcc/internal-fn.c:568
0xdf68da expand_mul_overflow
	../../gcc/internal-fn.c:1849
0xdf7f41 expand_arith_overflow
	../../gcc/internal-fn.c:2149
0xdf8317 expand_MUL_OVERFLOW
	../../gcc/internal-fn.c:2242
0xdfab1a expand_internal_call(internal_fn, gcall*)
	../../gcc/internal-fn.c:2778
0xdfab45 expand_internal_call(gcall*)
	../../gcc/internal-fn.c:2786
0xb42e0f expand_call_stmt
	../../gcc/cfgexpand.c:2569
0xb42e0f expand_gimple_stmt_1
	../../gcc/cfgexpand.c:3571
0xb42e0f expand_gimple_stmt
	../../gcc/cfgexpand.c:3737
0xb4486e expand_gimple_basic_block
	../../gcc/cfgexpand.c:5744
0xb4a4d6 execute
	../../gcc/cfgexpand.c:6357

The problem is that for __builtin_*_overflow{,_p} and/or -fsanitize=undefined and/or for the automatic discovery of constructs checking for overflow uses complex integer types and apparently that is just completely broken on msp430 and perhaps on other targets that have other similar partial int modes, I think that might be avr, bfin and m32c.

Try a simple testcase like:
typedef _Complex __int20 C;

C
foo (C x, C y)
{
  return x + y;
}

which ICEs the same.

Greetings,

I also ran into this problem when compiling newlib [snapshot 2.5.0.20170421] with a bootstrapping gcc using version 7.1.0 for the msp430-elf target.

$ msp430-elf-gcc -v
Using built-in specs.
COLLECT_GCC=msp430-elf-gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/msp430-elf/7.1.0/lto-wrapper
Target: msp430-elf
Configured with: ../configure --prefix=/usr --program-prefix=msp430-elf- --target=msp430-elf --host=x86_64-pc-linux-gnu --build=x86_64-pc-linux-gnu --disable-shared --disable-nls --disable-threads --enable-languages=c --enable-multilib --without-headers --with-newlib --with-local-prefix=/usr/msp430-elf --with-sysroot=/usr/msp430-elf --with-as=/usr/bin/msp430-elf-as --with-ld=/usr/bin/msp430-elf-ld --disable-libgomp
Thread model: single
gcc version 7.1.0 (GCC) 

The particular line that fails is (expanded):

 if ((nmemb >= ((size_t)1 << (sizeof(size_t) * 4)) || size >= ((size_t)1 << (sizeof(size_t) * 4))) &&
     nmemb > 0 && 
                 (0xfffffUL) 
                          / nmemb < size) {


Failure occurs at the comparison:

../../../../../../newlib/libc/stdlib/reallocarray.c:37:36: internal compiler error: in simplify_subreg, at simplify-rtx.c:6034
      nmemb > 0 && SIZE_MAX / nmemb < size) {

Is there any way to disable/bypass this particular behavior? Thank you.

Cheers,
Orlando.

Seeing this issue with a test case as simple as:

int
reallocarray(unsigned __int20 a, unsigned __int20 b)
{
 return 0xfffffUL / a < b;
}

This is a regression in GCC7/GCC8, GCC6 compiles it successfully. Started with the following commit:

Author: amonakov <amonakov@138bc75d-0d04-0410-961f-82ee72b054a4>
Date:   Mon May 30 14:37:02 2016 +0000

    match.pd: optimize unsigned mul overflow check
    
    gcc/
    2016-05-28  Alexander Monakov  <amonakov@ispras.ru>
                Marc Glisse  <marc.glisse@inria.fr>
    
            PR tree-optimization/71289
            * match.pd (-1 / B < A, A > -1 / B): New transformations.
    
    gcc/testsuite/
    2016-05-28  Alexander Monakov  <amonakov@ispras.ru>
    
            PR tree-optimization/71289
            * gcc.dg/pr71289.c: New test.
    
    
    
    git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/trunk@236882 138bc75d-0d04-0410-961f-82ee72b054a4

(In reply to Alexey Neyman from comment #4)
> Author: amonakov <amonakov@138bc75d-0d04-0410-961f-82ee72b054a4>
> Date:   Mon May 30 14:37:02 2016 +0000

This just exposed the bug.

Still broken in 7.2

What are the chances of this bug getting some love?

(In reply to Andrei Pushkin from comment #6)
> Still broken in 7.2
> 
> What are the chances of this bug getting some love?

I'm happy to report that I am back working on MSP430 GCC. However, I have some other startup items to complete before I can get back to fixing bugs, but this is top of my bug list.

Proposed patch: https://gcc.gnu.org/ml/gcc-patches/2018-01/msg01108.html

Author: law
Date: Wed Feb 14 07:21:11 2018
New Revision: 257653

URL: https://gcc.gnu.org/viewcvs?rev=257653&root=gcc&view=rev
Log:
2018-02-14  Jozef Lawrynowicz <jozefl.gcc@gmail.com>

	PR target/79242
	* machmode.def: Define a complex mode for PARTIAL_INT.
	* genmodes.c (complex_class): Return MODE_COMPLEX_INT for
	MODE_PARTIAL_INT.
	* doc/rtl.texi: Document CSPImode.
	* config/msp430/msp430.c (msp430_hard_regno_nregs): Add CPSImode
	handling.
	(msp430_hard_regno_nregs_with_padding): Likewise.

	PR target/79242
	* gcc.target/msp430/pr79242.c: New test.

Added:
    trunk/gcc/testsuite/gcc.target/msp430/pr79242.c
Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/config/msp430/msp430.c
    trunk/gcc/doc/rtl.texi
    trunk/gcc/genmodes.c
    trunk/gcc/machmode.def
    trunk/gcc/testsuite/ChangeLog

(In reply to Jeffrey A. Law from comment #9)
> Author: law
> Date: Wed Feb 14 07:21:11 2018
> New Revision: 257653
> 
> URL: https://gcc.gnu.org/viewcvs?rev=257653&root=gcc&view=rev
> Log:
> 2018-02-14  Jozef Lawrynowicz <jozefl.gcc@gmail.com>
> 
> 	PR target/79242
> 	* machmode.def: Define a complex mode for PARTIAL_INT.
> 	* genmodes.c (complex_class): Return MODE_COMPLEX_INT for
> 	MODE_PARTIAL_INT.
> 	* doc/rtl.texi: Document CSPImode.
> 	* config/msp430/msp430.c (msp430_hard_regno_nregs): Add CPSImode
> 	handling.
> 	(msp430_hard_regno_nregs_with_padding): Likewise.
> 
> 	PR target/79242
> 	* gcc.target/msp430/pr79242.c: New test.
> 
> Added:
>     trunk/gcc/testsuite/gcc.target/msp430/pr79242.c
> Modified:
>     trunk/gcc/ChangeLog
>     trunk/gcc/config/msp430/msp430.c
>     trunk/gcc/doc/rtl.texi
>     trunk/gcc/genmodes.c
>     trunk/gcc/machmode.def
>     trunk/gcc/testsuite/ChangeLog

Did this fix it?

(In reply to Eric Gallager from comment #10)
> (In reply to Jeffrey A. Law from comment #9)
> > Author: law
> > Date: Wed Feb 14 07:21:11 2018
> > New Revision: 257653
> > 
> > URL: https://gcc.gnu.org/viewcvs?rev=257653&root=gcc&view=rev
> > Log:
> > 2018-02-14  Jozef Lawrynowicz <jozefl.gcc@gmail.com>
> > 
> > 	PR target/79242
> > 	* machmode.def: Define a complex mode for PARTIAL_INT.
> > 	* genmodes.c (complex_class): Return MODE_COMPLEX_INT for
> > 	MODE_PARTIAL_INT.
> > 	* doc/rtl.texi: Document CSPImode.
> > 	* config/msp430/msp430.c (msp430_hard_regno_nregs): Add CPSImode
> > 	handling.
> > 	(msp430_hard_regno_nregs_with_padding): Likewise.
> > 
> > 	PR target/79242
> > 	* gcc.target/msp430/pr79242.c: New test.
> > 
> > Added:
> >     trunk/gcc/testsuite/gcc.target/msp430/pr79242.c
> > Modified:
> >     trunk/gcc/ChangeLog
> >     trunk/gcc/config/msp430/msp430.c
> >     trunk/gcc/doc/rtl.texi
> >     trunk/gcc/genmodes.c
> >     trunk/gcc/machmode.def
> >     trunk/gcc/testsuite/ChangeLog
> 
> Did this fix it?

The patch is currently only on trunk, but after back-porting it to gcc-7-branch, GCC now builds successfully for the msp430-elf target.
The original testcase also compiles successfully with the resulting toolchain.

Trunk still doesn't build for msp430-elf though, due to Bug 84406.

Fixed on gcc-7 branch as well.

Author: law
Date: Sat Feb 17 00:25:15 2018
New Revision: 257779

URL: https://gcc.gnu.org/viewcvs?rev=257779&root=gcc&view=rev
Log:
2018-02-16  Jozef Lawrynowicz <jozefl.gcc@gmail.com>

	PR target/79242
	* machmode.def: Define a complex mode for PARTIAL_INT.
	* genmodes.c (complex_class): Return MODE_COMPLEX_INT for
	MODE_PARTIAL_INT.
	* doc/rtl.texi: Document CSPImode.
	* config/msp430/msp430.c (msp430_hard_regno_nregs): Add CPSImode
	handling.
	(msp430_hard_regno_nregs_with_padding): Likewise.

	PR target/79242
	gcc.target/msp430/pr79242.c: New test.

Added:
    branches/gcc-7-branch/gcc/testsuite/gcc.target/msp430/pr79242.c
Modified:
    branches/gcc-7-branch/gcc/ChangeLog
    branches/gcc-7-branch/gcc/config/msp430/msp430.c
    branches/gcc-7-branch/gcc/doc/rtl.texi
    branches/gcc-7-branch/gcc/genmodes.c
    branches/gcc-7-branch/gcc/machmode.def
    branches/gcc-7-branch/gcc/testsuite/ChangeLog