Re: [PATCH] Allow MIPS call-saved-{4-6}.c tests to correctly run for micromips

[Richard Sandiford <rdsandiford at googlemail dot com>]

"Maciej W. Rozycki" <macro@linux-mips.org> writes:
> On Tue, 13 Jan 2015, Matthew Fortune wrote:
>
>> > >> I have tested this for both mips and micromips, and the tests now
>> > >> pass successfully.
>> > >> The ChangeLog and patch are below.
>> > >
>> > >  Hmm, instead of trying to avoid testing microMIPS code generation
>> > > just to satisfy the test suite I'd rather see the test cases updated
>> > > so that LWM/SWM register ranges are expected and accepted whenever
>> > > microMIPS code is produced.  These scan patterns can be made
>> > conditional.
>> > 
>> > FWIW I think Andrew's patch is correct.  If we want to test microMIPS
>> > output against micromips-specific regexps, we should add a separate test
>> > that forces micromips, so that it gets tested regardless of people's
>> > RUNTESTFLAGS.  Doing that shouldn't hold up Andrew's patch though.
>
>  Taking care that the default compilation mode does not conflict (e.g. 
> MIPS16, incompatible) and taking any exceptions into account (e.g. n64, 
> unsupported) I presume, right?

mips.exp sorts that out for you.  Adding "-mmicromips" or "(-micromips)"
to dg-options forces (or at least is supposed to force) the overall flags
to be compatible with microMIPS.

The aim of mips.exp is avoid skipping tests whereever possible.  If
someone runs the testsuite with -mips16 and we have a -micromips test,
it's better to remove -mips16 for that test than to skip the test entirely.

>> I was going to suggest a follow up patch to add copies of the three tests
>> as Richard suggests. I haven't yet done a micromips run of the testsuite
>> to check for any other issues like this but I suspect problems are limited
>> to the tests that I recently added.
>
>  Please always try to test changes reasonably, i.e. at least o32, 
> o32/MIPS16, o32/microMIPS, n32, n64, and then Linux and ELF if applicable, 
> plus any options that may be relevant, unless it is absolutely clear 
> ABI/ISA variations do not matter for a change proposed.

TBH this seems a bit much.  On the one hand it's more testing than you'd
get for almost any other target, but on the other it leaves out important
differences like MIPS I vs MIPS II vs MIPS 32, MIPS III vs MIPS IV vs MIPS64,
r1 vs. r2 vs. r6, Octeon vs. Loongson vs. vanilla, DSP vs. no DSP, etc.
I think we just have to accept that there are so many possible
combinations that we can't test everything that's potentially relevant.
I think it's more useful to be flexible than prescribe a particular list.

Having everyone test the same multilib combinations on the same target
isn't necessarily a good thing anyway.  Diversity in testing (between
developers) is useful too.

Thanks,
Richard