I think it is some libstdc++-v3 header change that triggered it lately, but don't have setup where I can easily bisect that.
When I try to bisect with preprocessed xtreme-header-4_a.ii
and
-quiet xtreme-header-4_a.ii -quiet -Wno-long-long -std=c++17 -fmodule-header -o xtreme-header-4_a.s
options to cc1plus, the ICE started with
r11-6083-gb7dfc2074c78415d451eb34d1608016c80b1c41a

Based on the ICE details (on __data member of anonymous struct inside of
__from_chars_alnum_to_val_table) I think this started with:
r12-8175-ga54137c88061c7495728fc6b8dfd0474e812b2cb

Small testcase:
pr105297.h:

constexpr auto
foo ()
{
  struct S { unsigned char d[1u << __CHAR_BIT__] = {}; } t;
  return t;
}

template <int N>
unsigned char
bar (unsigned char x)
{
  static constexpr auto t = foo ();
  return t.d[x];
}

./cc1plus -o pr105297.{s,h} -fmodule-header

If we want a quick work-around, I think:
--- libstdc++-v3/include/std/charconv.jj	2022-04-19 07:20:56.772166410 +0200
+++ libstdc++-v3/include/std/charconv	2022-04-20 16:27:47.971314921 +0200
@@ -407,6 +407,10 @@ namespace __detail
       return true;
     }
 
+  struct __from_chars_alnum_to_val_table_type {
+    unsigned char __data[1u << __CHAR_BIT__] = {};
+  };
+
   // Construct and return a lookup table that maps 0-9, A-Z and a-z to their
   // corresponding base-36 value and maps all other characters to 127.
   constexpr auto
@@ -420,7 +424,7 @@ namespace __detail
       = { 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J',
 	  'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T',
 	  'U', 'V', 'W', 'X', 'Y', 'Z' };
-    struct { unsigned char __data[1u << __CHAR_BIT__] = {}; } __table;
+    struct __from_chars_alnum_to_val_table_type __table;
     for (auto& __entry : __table.__data)
       __entry = 127;
     for (int __i = 0; __i < 10; ++__i)
or so would do it.

Consistent with Jakub's comment#2, also appearing for cris-elf (between r12-8173-ge580f81d22d611 and r12-8175-ga54137c88061c7), and apparently all other targets judging from today's gcc-testresults posts.

Patrick, I suggest we go with Jakub's suggestion in comment 4, but PTAL.

(In reply to Jonathan Wakely from comment #6)
> Patrick, I suggest we go with Jakub's suggestion in comment 4, but PTAL.

LGTM, it seems to be the simplest workaround.  I also tried replacing the NSDMI but that seems to prevent the ICE only for the reduced testcase in comment #3.

I shall test/push that fix (unless Jakub beats me to it :))

(In reply to Patrick Palka from comment #7)
> (In reply to Jonathan Wakely from comment #6)
> > Patrick, I suggest we go with Jakub's suggestion in comment 4, but PTAL.
> 
> LGTM, it seems to be the simplest workaround.  I also tried replacing the
> NSDMI but that seems to prevent the ICE only for the reduced testcase in
> comment #3.

Note, the original reduced testcase didn't have the " S" part in there,
i.e. used anonymous struct.  Is that what matters for the NSDMI?
I

(In reply to Jakub Jelinek from comment #9)
> (In reply to Patrick Palka from comment #7)
> > (In reply to Jonathan Wakely from comment #6)
> > > Patrick, I suggest we go with Jakub's suggestion in comment 4, but PTAL.
> > 
> > LGTM, it seems to be the simplest workaround.  I also tried replacing the
> > NSDMI but that seems to prevent the ICE only for the reduced testcase in
> > comment #3.
> 
> Note, the original reduced testcase didn't have the " S" part in there,
> i.e. used anonymous struct.  Is that what matters for the NSDMI?

Interestingly that doesn't seem to make a difference.  What seems to matter is whether the constexpr function modifies the CONSTRUCTOR that it returns:

constexpr auto foo() {
  struct S { int d; } t = {};
  t.d = 0; // doesn't ICE if this line is commented out
  return t;
}

template<int>
int bar() {
  constexpr auto t = foo();
  return 0;
}

(In reply to Patrick Palka from comment #10)
> 
> Interestingly that doesn't seem to make a difference.  What seems to matter
> is whether the constexpr function modifies the CONSTRUCTOR that it returns:
> 
> constexpr auto foo() {
>   struct S { int d; } t = {};
>   t.d = 0; // doesn't ICE if this line is commented out
>   return t;
> }
> 
> template<int>
> int bar() {
>   constexpr auto t = foo();
>   return 0;
> }

Right, it is weird. Some PRs on Xtreme-* failure (including ICE) were also reported before. e.g. PR100052, PR101853, PR99910.  As commented in those PRs, these may be random failures, and changes in headers that could expose the ICE.
I'm also wondering if this may be an issue hidden inside somewhere (GC?).

The master branch has been updated by Patrick Palka <ppalka@gcc.gnu.org>:

https://gcc.gnu.org/g:1e6c0e69af8da436e1d1d2d23d8c38410d78ecf2

commit r12-8214-g1e6c0e69af8da436e1d1d2d23d8c38410d78ecf2
Author: Patrick Palka <ppalka@redhat.com>
Date:   Thu Apr 21 08:34:59 2022 -0400

    libstdc++: Work around modules ICE in <charconv> [PR105297]
    
    This makes the initializer for __table in __from_chars_alnum_to_val
    dependent in an artificial way, which works around the reported modules
    testsuite ICE by preventing the compiler from evaluating the initializer
    parse time.
    
    Compared to the alternative workaround of using a non-local class type
    for __table, this workaround has the advantage of slightly speeding up
    compilation of <charconv>, since now the table won't get built (via
    constexpr evaluation) until the integer std::from_chars overload is
    instantiated.
    
            PR c++/105297
            PR c++/105322
    
    libstdc++-v3/ChangeLog:
    
            * include/std/charconv (__from_chars_alnum_to_val): Make
            initializer for __table dependent in an artificial way.

(In reply to Jiu Fu Guo from comment #11)
> (In reply to Patrick Palka from comment #10)
> > 
> > Interestingly that doesn't seem to make a difference.  What seems to matter
> > is whether the constexpr function modifies the CONSTRUCTOR that it returns:
> > 
> > constexpr auto foo() {
> >   struct S { int d; } t = {};
> >   t.d = 0; // doesn't ICE if this line is commented out
> >   return t;
> > }
> > 
> > template<int>
> > int bar() {
> >   constexpr auto t = foo();
> >   return 0;
> > }
> 
> Right, it is weird. Some PRs on Xtreme-* failure (including ICE) were also
> reported before. e.g. PR100052, PR101853, PR99910.  As commented in those
> PRs, these may be random failures, and changes in headers that could expose
> the ICE.
> I'm also wondering if this may be an issue hidden inside somewhere (GC?).

In this case I suspect it's just a bug in the modules code, I opened PR105322 to track it.

Fixed, hopefully.

(In reply to Patrick Palka from comment #13)
> (In reply to Jiu Fu Guo from comment #11)
> > (In reply to Patrick Palka from comment #10)
> > > 
> > > Interestingly that doesn't seem to make a difference.  What seems to matter
> > > is whether the constexpr function modifies the CONSTRUCTOR that it returns:
> > > 
> > > constexpr auto foo() {
> > >   struct S { int d; } t = {};
> > >   t.d = 0; // doesn't ICE if this line is commented out
> > >   return t;
> > > }
> > > 
> > > template<int>
> > > int bar() {
> > >   constexpr auto t = foo();
> > >   return 0;
> > > }
> > 
> > Right, it is weird. Some PRs on Xtreme-* failure (including ICE) were also
> > reported before. e.g. PR100052, PR101853, PR99910.  As commented in those
> > PRs, these may be random failures, and changes in headers that could expose
> > the ICE.
> > I'm also wondering if this may be an issue hidden inside somewhere (GC?).
> 
> In this case I suspect it's just a bug in the modules code, I opened
> PR105322 to track it.

Oh, thanks!  This failure seems only about the module code on 'struct member cross functions'.