Expensive selftests (was: 'hash_map<tree, hash_map<tree, tree>>')

[Thomas Schwinge]

Hi!

On 2021-08-16T14:10:00-0600, Martin Sebor <msebor@gmail.com> wrote:
> On 8/16/21 6:44 AM, Thomas Schwinge wrote:
>> [...], to document the current behavior, I propose to
>> "Add more self-tests for 'hash_map' with Value type with non-trivial
>> constructor/destructor", see attached.  OK to push to master branch?
>> (Also cherry-pick into release branches, eventually?)

(Attached again, for easy reference.)

> Adding more tests sounds like an excellent idea.  I'm not sure about
> the idea of adding loopy selftests that iterate as many times as in
> the patch (looks like 1234 times two?)

Correct, and I agree it's a sensible concern, generally.

The current 1234 times two iterations is really arbitrary (should
document that in the test case), just so that we trigger a few hash table
expansions.

For 'selftest-c', we've got originally:

    -fself-test: 74775 pass(es) in 0.309299 seconds
    -fself-test: 74775 pass(es) in 0.366041 seconds
    -fself-test: 74775 pass(es) in 0.356663 seconds
    -fself-test: 74775 pass(es) in 0.355009 seconds
    -fself-test: 74775 pass(es) in 0.367575 seconds
    -fself-test: 74775 pass(es) in 0.320406 seconds

..., and with my changes we've got:

    -fself-test: 94519 pass(es) in 0.327755 seconds
    -fself-test: 94519 pass(es) in 0.369522 seconds
    -fself-test: 94519 pass(es) in 0.355531 seconds
    -fself-test: 94519 pass(es) in 0.362179 seconds
    -fself-test: 94519 pass(es) in 0.363176 seconds
    -fself-test: 94519 pass(es) in 0.318930 seconds

So it really seems to be all in the noise?

Yet:

> Selftests run each time GCC
> builds (i.e., even during day to day development).  It seems to me
> that it might be better to run such selftests only as part of
> the bootstrap process.

I'd rather have thought about a '--param self-test-expensive' (or
similar), and then invoke the selftests via a new
'gcc/testsuite/selftests/expensive.exp' (or similar).

Or, adapt 'gcc/testsuite/gcc.dg/plugin/expensive_selftests_plugin.c',
that is, invoke them via the GCC plugin mechanism, which also seems to be
easy enough?

I don't have a strong opinion about where/when these tests get run, so
will happily take any suggestions.


Grüße
 Thomas


-----------------
Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht München, HRB 106955
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Add-more-self-tests-for-hash_map-with-Value-type-wit.patch
Type: text/x-diff
Size: 4954 bytes
Desc: not available
URL: <https://gcc.gnu.org/pipermail/gcc/attachments/20210817/0845a9aa/attachment.bin>