[Bug bootstrap/66038] [5 regression] (stage 2) build/genmatch segfaults in operand::gen_transform (gcc/hash-table.h:223)

[dougmencken at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66038

--- Comment #11 from Douglas Mencken <dougmencken at gmail dot com> ---
Causing commit found.

It is r218976 (e2afa5c10fd41fe708959121f373fcb5435ef5d6). With reverse-applied 
r218976's patch, 5.1.0 even reaches "Bootstrap comparison failure!‘‘ ;)

Maybe patch's author [ Author: hubicka ] can help here.

I see in patch:

-hashval_t
-hash_table_mod1 (hashval_t hash, unsigned int index)
-{
-  const struct prime_ent *p = &prime_tab[index];
-#ifdef UNSIGNED_64BIT_TYPE
-  if (sizeof (hashval_t) * CHAR_BIT <= 32)
-    return mul_mod (hash, p->prime, p->inv, p->shift);
-#endif
-  return hash % p->prime;
-}

+inline hashval_t
+hash_table_mod1 (hashval_t hash, unsigned int index)
+{
+  const struct prime_ent *p = &prime_tab[index];
+  gcc_checking_assert (sizeof (hashval_t) * CHAR_BIT <= 32);
+    return mul_mod (hash, p->prime, p->inv, p->shift);
+}

Before, when "sizeof (hashval_t) * CHAR_BIT <= 32”, hash % p->prime was
returned. After, <= 32 triggers assert (--> failure).

I suggest something like

inline hashval_t
hash_table_mod1 (hashval_t hash, unsigned int index)
{
  const struct prime_ent *p = &prime_tab[index];
  if (sizeof (hashval_t) * CHAR_BIT <= 32)
    return mul_mod (hash, p->prime, p->inv, p->shift);
  else
    return hash % p->prime;
}