Re: *_data_map_rand.cc timeout on mips

[Benjamin Kosnik <bkoz at redhat dot com>]

>   Overall, I don't see a way to reduce the product (1) * (2) * (3) -
> the number of containers is minimal (containers which didn't show any
> advantages were already deleted), regression tests should check for
> everything, and 3 is up to the compiler (if there's a way to write the
> same code that makes for lighter compilation - I'm not aware of it).

Agreed.

I've been playing around, trying to find a way to speed up compilation.

However, I cannot figure out a way to do this. The actual
pre-compiling/pre-processing part of pb_ds testcases is not the thing
causing problems.

It's the compile-time computation once all the includes are set.

Solutions that I see are:

1) don't do this heavy computation at all
2) do less of it, or break up into many smaller bits of computation
3) make the compiler go faster
4) figure out better coding styles that make the compiler go faster
5) allow the compiler to take more time
6) don't do this kind of test on slow machines.

All are less than perfect.

I just checked in a patch for #5. I'm hoping #6 will not be necessary.

Perhaps we can get feedback on #4 from Mark or Jason.

>   Instead, it's possible to do one of the following:
> 1. Make more *_*_map_rand.cc files, each testing a specific container
> and policy combination. This would not reduce the product (1) * (2) *
> (3) (because the total compilation time would be the same), but it
> would mean that each compilation would be shorter. I think there's a
> better solution.

I hope for a better solution as well.

> 2. Randomized tests are strong in the sense that if there is a
> problem, the chances that it will go undetected decreases over time.
> Why not make smaller tests then (as in 1), but decide randomly which
> ones to compile and run each time. In fact, just like the typelists
> now expand to cover all possible policy combinations at compile time,
> I can write the same code in python to generate a random subset of
> tests at runtime (that is, the *_*map_rand.cc contents would be
> generated on the fly). It's possible to object that not all platforms
> would test the same things at the same time, but since the tests are
> randomized - they're not doing so anyway.

Ugh.

Well, this adds a bunch more complexity to something that is already
quite complex.

In addition, this would mean that testresults on the same machine would
not be the same, run after run. Since we're going for consistency, this
would be a problem.

My plan is:
1) increase timeouts
2) switch to stdtr1c++.h PCH for the testsuite

After this round of changes, we can examine remaining fails.

-benjamin