ACATS legal status cleared by FSF
dewar@gnat.com
dewar@gnat.com
Fri Dec 7 19:12:00 GMT 2001
Note that just *reading* B tests to see if the output is correct is a
very difficult task, one that only someone with quite a bit of ACVC/ACATS
validation experience can do. A formal validation run using these tests
often involves several days of painstaking manual work by someone who
is an expert in the B tests to assure compliance.
Consider various possibilities:
1. You must not make changes to the compiler that change the B tests baselines.
This is far too restrictive, it would forbid even fixing a spelling error.
In practice fixing a bug in one part of the compiler can often change
an error message elsewhere in one of the B tests (by implementing
better error recovery). Most often such changes are for the better.
2. If the B test baseline changes, verify the baseline change and adjust the
baseline.
This would be fine, except that the verification process typically
requires someone with very good expertise in Ada semantics, AND
very good familiarity with the ACATS test suite. There are not
many such people in the world.
A huge effort from any Ada vendor goes into making sure that the compiler
passes all the B tests. Many have actively questioned the value of this
effort in the past, but if you want to formally validate you have no
choice. But this does not necessarily mean that these are well chosen
tests from the point of view of our use here at gnu.org.
More information about the Gcc
mailing list