ACATS legal status cleared by FSF

[<guerby at acm dot org>]

I just received confirmation from FSF that ACATS DFAR legalese means
that we can do whatever we want with it. In this email I make a few
descriptions and proposals, they start by [N].

The executable part of ACATS (formaly A, C, D, E and L tests - see
below) comprises 2500 source files, summing to 15 MB and providing
around 2300 executable tests. Each source file has a DFAR legalese
header - attached after my signature - and a description of the test.

>From the ACATS documentation

<<
Class A tests check for acceptance (compilation) of language
constructs that are expected to compile without error.

Class B tests  check that illegal  constructs are recognized  and
treated as fatal  errors. They are  not expected to  successfully
compile, bind, or execute.

Class C tests  check that executable  constructs are  implemented
correctly and produce expected results. These tests are  expected
to  compile,  bind,   execute  and  report   "PASSED"  or   "NOT-
APPLICABLE".  Each   class  C   test  reports   "PASSED",   "NOT-
APPLICABLE", or "FAILED" based on  the results of the  conditions
tested.

Class D tests check that implementations perform exact arithmetic
on large literal  numbers. These tests  are expected to  compile,
bind, execute and report "PASSED". Each test reports "PASSED"  or
"FAILED" based on the conditions tested. Some implementations may
report errors at compile  time for some of  them, if the  literal
numbers exceed compiler limits.

Class E tests check for constructs that may require inspection to
verify. They have special grading criteria that are stated within
the test source.

Class L tests check that all  library unit dependencies within  a
program are  satisfied  before  the  program  can  be  bound  and
executed, that  circularity  among  units is  detected,  or  that
pragmas  that  apply  to   an  entire  partition  are   correctly
processed.
>>

All ACATS tests are identifed by a 7 characters key which is more or
less composed from Class + Reference Manual Chapter + RM Section + Key.

All the files for a test begin by this 7 character key, most tests
have only one file, but some have more than one.  An ACATS source file
can contain multiple compilation unit, to run them through GNAT we
first need to "gnatchop" them so that each unit ends up with one file
with the name GNAT expects by default, then we have to guess what is
the file containing the main routine, "gnatmake" it and run it.
This increases the count of source files from 2500 to 4100.

To avoid preprocessing and script-like machinery overhead, I propose
that we commit the ACATS sources directly in the form expected by
GNAT, so we end up just having a list of main unit names, and so a
simple minded loop of "gnatmake x; run x" will be the only thing
needed to run. 

[1] Any objection?

[2] Do we want to keep some subdirectories, if so what granularity, or
all in one dir is okay (something like 4000 files)?

[3] Some file names will be up to 56 characters, okay? (87% have less
or equal than 14.)

[4] What should be the top directory in the gcc tree for the ACATS
test suite?  gcc/gcc/testsuite/ada/acats or a top level directory or
something else? I assume we want ACATS somewhat separated from other
Ada tests generated by the GCC project, this will facilitate
maintenance.

[5] IMHO it is best if the form in which we commit the sources of the
ACATS tests is as separated as possible from the testsuite harness
technology at first, just sources, README and a list of test names.
Okay?

There are convoluted tests trying to see if the compiler conforms to
the Ada compilation model initially suggested by the RM, but as RMS
invented a new way of reading this and that GNAT does follow it (and
modern proprietary compiler do the same), these tests are of no value,
I propose we just drop them. I'll of course document all such tests
dropped and why.

[6] Okay?

The B tests require a lot of maintainance (hundreds of pages of
changes each time you improve a message, split of files with too many
errors, etc...), and have no value for the backend, may be someone
will volunteer the packaging, but not me. I assume ACT dedicates
someone to this task anyway :).

[7] I can help if someone wants to take over this, but ACT has probably
to provide an initial baseline of splits and scripts, so please check
with them first.

Here is a rough count of executable tests by class and chapter (lower
bound, some are omitted):

   4 ; cz Check that the test reporting stuff works
  75 ; a  
  34 ; c2 Lexical Elements
 351 ; c3 Declarations and Types
 339 ; c4 Names and Expressions
  95 ; c5 Statements
  81 ; c6 Subprograms 
  51 ; c7 Packages
 140 ; c8 Visibility Rules
 255 ; c9 Tasks and Synchronization
  74 ; ca Program Structure and Compilation Issues
  43 ; cb Exceptions
 117 ; cc Generic Units
 173 ; cd Representation Issues
 268 ; ce Predefined Language Environment (Ada 83)
  87 ; cxa Predefined Language Environment (Ada 95)
  30 ; cxb Interface to Other Languages
  13 ; cxc Systems Programming
  38 ; cxd Real-Time Systems
   1 ; cxe Distributed Systems 
  20 ; cxf Information Systems
  29 ; cxg Numerics
   4 ; cxh Safety and Security
   4 ; d  
  11 ; e  
  26 ; l  

I have a patch applying to all tests using the delay statement in
order to be able to rescale them (their default value in second is
absurd for modern machines), the patch is composed of hunks like:

+with Impdef;
 PACKAGE BODY C94005B_PKG IS
 
      TASK BODY TT IS
@@ -59,7 +59,7 @@
           ACCEPT E (I : INTEGER) DO
                LOCAL := I;
           END E;
-          DELAY 60.0;    -- SINCE THE PARENT UNIT HAS HIGHER PRIORITY
+          DELAY 60.0 * Impdef.One_Second;    -- SINCE THE PARENT UNIT HAS HIGHER PRIORITY
                          -- AT THIS POINT, IT WILL RECEIVE CONTROL AND
                          -- TERMINATE IF THE ERROR IS PRESENT.
           GLOBAL := LOCAL;

[8] I propose to commit directly with the patch applied, I'll of course
commit and maintain a list of tests that were modified and why, any
objection?

[9] Before doing the commit, I'll need a place where to put the
various prototype tarballs to reflect discussions and proposals, could
someone give me access to an ftp server somewhere?

[10] The first harness prototype will be ultra simple and should allow
anyone to play with it so we can make progress on portability and
features later on by patches on stuff in CVS the regular GCC way
instead of by generating a 20 MB tarball each time we change
something.

The upstream ACATS is maintained using CVS, web documents and
interface from <http://www.ada-auth.org/>. There's a very low volume
public mailing list.

[12] Do we want to scrap the legalese and replace it by something else?
Do we want to identify changes or extension made within the GCC
project, and if so how?

[13] I apply for maintainership of this stuff once commited, and will
keep it in synch with the change made upstream.

-- 
Laurent Guerby <guerby@acm.org>

--                             Grant of Unlimited Rights
--
--     Under contracts F33600-87-D-0337, F33600-84-D-0280, MDA903-79-C-0687,
--     F08630-91-C-0015, and DCA100-97-D-0025, the U.S. Government obtained 
--     unlimited rights in the software and documentation contained herein.
--     Unlimited rights are defined in DFAR 252.227-7013(a)(19).  By making 
--     this public release, the Government intends to confer upon all 
--     recipients unlimited rights  equal to those held by the Government.  
--     These rights include rights to use, duplicate, release or disclose the 
--     released technical data and computer software in whole or in part, in 
--     any manner and for any purpose whatsoever, and to have or permit others 
--     to do so.
--
--                                    DISCLAIMER
--
--     ALL MATERIALS OR INFORMATION HEREIN RELEASED, MADE AVAILABLE OR
--     DISCLOSED ARE AS IS.  THE GOVERNMENT MAKES NO EXPRESS OR IMPLIED 
--     WARRANTY AS TO ANY MATTER WHATSOEVER, INCLUDING THE CONDITIONS OF THE
--     SOFTWARE, DOCUMENTATION OR OTHER INFORMATION RELEASED, MADE AVAILABLE 
--     OR DISCLOSED, OR THE OWNERSHIP, MERCHANTABILITY, OR FITNESS FOR A
--     PARTICULAR PURPOSE OF SAID MATERIAL.