Re: [PATCH][RFC] Sanitize equals and hash functions in hash-tables.

[Jason Merrill <jason at redhat dot com>]

On 5/13/19 3:41 AM, Martin Liška wrote:
> On 11/8/18 9:56 AM, Martin Liška wrote:
> > On 11/7/18 11:23 PM, Jeff Law wrote:
> > > On 10/30/18 6:28 AM, Martin Liška wrote:
> > > > On 10/30/18 11:03 AM, Jakub Jelinek wrote:
> > > > > On Mon, Oct 29, 2018 at 04:14:21PM +0100, Martin Liška wrote:
> > > > > > +hashtab_chk_error ()
> > > > > > +{
> > > > > > +  fprintf (stderr, "hash table checking failed: "
> > > > > > +	   "equal operator returns true for a pair "
> > > > > > +	   "of values with a different hash value");
> > > > > BTW, either use internal_error here, or at least if using fprintf
> > > > > terminate with \n, in your recent mail I saw:
> > > > > ...different hash valueduring RTL pass: vartrack
> > > > >                      ^^^^^^
> > > > Sure, fixed in attached patch.
> > > >
> > > > Martin
> > > >
> > > > > > +  gcc_unreachable ();
> > > > > > +}
> > > > > 	Jakub
> > > >
> > > > 0001-Sanitize-equals-and-hash-functions-in-hash-tables.patch
> > > >
> > > >  From 0d9c979c845580a98767b83c099053d36eb49bb9 Mon Sep 17 00:00:00 2001
> > > > From: marxin <mliska@suse.cz>
> > > > Date: Mon, 29 Oct 2018 09:38:21 +0100
> > > > Subject: [PATCH] Sanitize equals and hash functions in hash-tables.
> > > >
> > > > ---
> > > >   gcc/hash-table.h | 40 +++++++++++++++++++++++++++++++++++++++-
> > > >   1 file changed, 39 insertions(+), 1 deletion(-)
> > > >
> > > > diff --git a/gcc/hash-table.h b/gcc/hash-table.h
> > > > index bd83345c7b8..694eedfc4be 100644
> > > > --- a/gcc/hash-table.h
> > > > +++ b/gcc/hash-table.h
> > > > @@ -503,6 +503,7 @@ private:
> > > >   
> > > >     value_type *alloc_entries (size_t n CXX_MEM_STAT_INFO) const;
> > > >     value_type *find_empty_slot_for_expand (hashval_t);
> > > > +  void verify (const compare_type &comparable, hashval_t hash);
> > > >     bool too_empty_p (unsigned int);
> > > >     void expand ();
> > > >     static bool is_deleted (value_type &v)
> > > > @@ -882,8 +883,12 @@ hash_table<Descriptor, Allocator>
> > > >     if (insert == INSERT && m_size * 3 <= m_n_elements * 4)
> > > >       expand ();
> > > >   
> > > > -  m_searches++;
> > > > +#if ENABLE_EXTRA_CHECKING
> > > > +    if (insert == INSERT)
> > > > +      verify (comparable, hash);
> > > > +#endif
> > > >   
> > > > +  m_searches++;
> > > >     value_type *first_deleted_slot = NULL;
> > > >     hashval_t index = hash_table_mod1 (hash, m_size_prime_index);
> > > >     hashval_t hash2 = hash_table_mod2 (hash, m_size_prime_index);
> > > > @@ -930,6 +935,39 @@ hash_table<Descriptor, Allocator>
> > > >     return &m_entries[index];
> > > >   }
> > > >   
> > > > +#if ENABLE_EXTRA_CHECKING
> > > > +
> > > > +/* Report a hash table checking error.  */
> > > > +
> > > > +ATTRIBUTE_NORETURN ATTRIBUTE_COLD
> > > > +static void
> > > > +hashtab_chk_error ()
> > > > +{
> > > > +  fprintf (stderr, "hash table checking failed: "
> > > > +	   "equal operator returns true for a pair "
> > > > +	   "of values with a different hash value\n");
> > > > +  gcc_unreachable ();
> > > > +}
> > > I think an internal_error here is probably still better than a simple
> > > fprintf, even if the fprintf is terminated with a \n :-)
> >
> > Fully agree with that, but I see a lot of build errors when using internal_error.
> >
> > > The question then becomes can we bootstrap with this stuff enabled and
> > > if not, are we likely to soon?  It'd be a shame to put it into
> > > EXTRA_CHECKING, but then not be able to really use EXTRA_CHECKING
> > > because we've got too many bugs to fix.
> >
> > Unfortunately it's blocked with these 2 PRs:
> > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87845
> > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87847
>
> Hi.
>
> I've just added one more PR:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90450
>
> I'm sending updated version of the patch that provides a disablement for the 3 PRs
> with a new function disable_sanitize_eq_and_hash.
>
> With that I can bootstrap and finish tests. However, I've done that with a patch
> limits maximal number of checks:
>
> diff --git a/gcc/hash-table.h b/gcc/hash-table.h
> index dc24fea6405..57564914e31 100644
> --- a/gcc/hash-table.h
> +++ b/gcc/hash-table.h
> @@ -1027,7 +1027,7 @@ void
>   hash_table<Descriptor, Lazy, Allocator>
>   ::verify (const compare_type &comparable, hashval_t hash)
>   {
> -  for (size_t i = 0; i < m_size; i++)
> +  for (size_t i = 0; i < MIN (m_size, 1000); i++)
>       {
>         value_type *entry = &m_entries[i];
>         if (!is_empty (*entry) && !is_deleted (*entry)
>
> Without that it would be probably terribly slow. Moreover, one probably does not want
> that with an extra checking, but with an extra-extra checking. Ideas about where to enable
> it?

As a --param?  I use --param ggc-min-heapsize=0 --param ggc-min-expand=0 
to catch GC issues, and this seems similar.

> +  /* FIXME: enable sanitization */

Please add PR numbers to these comments.

> +#if ENABLE_EXTRA_CHECKING

The documentation describes this flag as

> @samp{extra} adds for @samp{misc} checking extra checks that might affect
> code generation and should therefore not differ between stage1 and later
> stages.

which doesn't seem to apply here.

Jason