ACATS

[Joseph S. Myers]

On Thu, 6 Dec 2001 guerby@acm.org wrote:

> self: I'm no CVS specialist, keeping things in ACATS original form
> will require maintaining piles of dubious scripts at best, I don't
> want to do that myself, I by far prefer the hand crafted approach, and
> have things directly in GCC usable form in CVS.

The point of using the vendor branch is to have *both* the original form
(on the vendor branch) and the GCC version (on the mainline) in CVS - and
when a new version is imported onto the vendor branch, CVS should be able
to help with a lot of the work of merging onto the mainline.  The CVS
experts here can probably provide step-by-step instructions for doing this
and indications of the pitfalls.

> Geoff Keating: You do plan to use dejagnu to run the tests, correct?
> 
> self: My initial tarball will only contain readily compilable sources,
> a list of test main names and attributes as described above,
> documentation on the test to run and omitted, and a minimalistic
> Bourne shell script to run the thing.  From there, I assume DejaGNU
> experts will volunteer or guide volunteers on how to best integrate
> this in the existing Makefile + dg scripts. I must admit I'm not
> really impressed by the existing testing framework, and I believe and
> I can do much better with not that much effort, I'll propose something
> later on, but I don't want this to delay inclusion of ACATS in usable
> form in the GCC project, and I certainly don't want to stop people
> that would volunteer on integrating ACATS and existing DejaGNU.
> People will also have to decide what is officially part of the
> mandatory testing process.

I think DejaGNU must be a requirement for this to be part of the standard
"make check".  As well as providing support for testing cross-compilers in
many different environments, both simulators and target boards, many
people have scripts that use DejaGNU-format .sum files to check for
regressions and analyse results.  For example, that's what the automated
regression checker uses, and it must be desirable for the regression
checker to run these tests.

-- 
Joseph S. Myers
jsm28@cam.ac.uk