Re: GCC 3.2

[Janis Johnson <janis187 at us dot ibm dot com>]

On Tue, Aug 20, 2002 at 09:31:15PM -0700, Benjamin Kosnik wrote:
> 
> Just an update for those not monitoring the libstdc++ list.
> 
> There's been a discussion of this, and a partial implementation by
> Ulrich, Phil, and myself. It looks like initial implementation is here
> 
> http://gcc.gnu.org/ml/libstdc++/2002-08/msg00190.html
> 
> Please comment if there are any bright ideas on how to make this better.

Benjamin,

You are focusing on libstdc++ testing rather than compiler and runtime
support, is that correct?

I've been trying to summarize what needs to be tested and what
approaches can be used for that testing, with the goal of figuring out
which part of this I can help with.  Here are my notes, based on your
document at http://gcc.gnu.org/onlinedocs/libstdc++/abi.txt, the C++
ABI, Intel's C++ ABI test suite, the LSB mail archives and FAQ, and
messages from several people in the GCC mailing lists.

Janis


Things to test

  compiler and runtime support covered by the C++ ABI:
    data layout
      layout of class and structure elements
      alignment of variables
      pointer to data member
      pointer to member function
      virtual table layout and alignment
      array operator "new" cookies
      initialization guard variables
      run-time type information (RTTI)
    function calling conventions and APIs
      stack unwind
      argument passing
      virtual function
      demangler API
      constructor and destructor APIs
    exception handling interfaces
    linkage and object files
      name mangling
      linkage of inline functions, vtables
      unwind table location
    other
      template instantiation
      runtime arithmetic support

  C++ Standard Library:
    namespaces
    sizes and layout of externally visible classes and structures
    signatures of externally visible functions

outside the scope of GCC testing efforts:

  cross-platform testing (including different releases of the same OS)
  testing with different linkers and/or dynamic linkers
  startup files?

Approaches to testing C++ and libstdc++ compatibility, ABI conformance

libstdc++ consistency
  http://gcc.gnu.org/ml/libstdc++/2002-08/msg00190.html

  covers:
    libstdc++ consistency from release to release

  advantages:

  disadvantages:

Intel's C++ ABI test suite:
  http://developer.intel.com/software/products/opensource/

  covers:
    compiler and runtime support covered by the C++ ABI

  advantages:
    exists, available

  disadvantages:
    limited to IA-64?
    large granularity for failures

gcc/testsuite/consistency.vlad
  covers:
    layout of class and structure elements

  advantages:
    already exists
    fits into the existing GCC testsuite

  disadvantages:
    need to establish a baseline for each platform

LSB conformance test suite
  covers:
    not sure yet

  advantages:
    will be developed by the LSB team

  disadvantages:
    only applies to platforms covered by the LSB

gdb test suite (suggested by Jim Wilson)
  covers (indirectly):
    layout of structure elements
    argument passing
    vtable layout
    name mangling

  advantages:
    easy to use for anyone who runs the GCC testsuite
    available on many platforms (anywhere gdb is available)

  disadvantages:
    can't be used on platforms where gdb isn't available

GCC test suites with different version of libstdc++
  covers:
    compiler and runtime support used by libstdc++
    libstdc++ consistency from release to release

  advantages:
    easy to set up

  disadvantages:
    need to recognize test failures due to changes in the test suite

special-purpose self-checking tests in GCC testsuite
  covers:
    whatever we want

  advantages:
    part of every testsuite run

  disadvantages:
    limited to what the testsuite supports: single file per test, no
    shared libraries, use of a single compiler

split tests to compile pieces with two compilers
  http://gcc.gnu.org/ml/gcc/2002-08/msg00712.html

  covers:
    anything for the compiler and runtime support library

  advantages:
    has the potential to uncover unexpected problems that wouldn't be
    detected by other testing

  disadvantages:
    doesn't yet exist

    won't easily fit into the GCC testsuite
      need to have two compilers available when tests are run or else
      save baseline object files for each platform (.o, .so, .a)

      building static libraries and shared objects is OS-dependent

Glossary

  C++ ABI: An Application Binary Interface for C++ programs that was
    designed to be specific to 64-bit Itanium but that also includes
    generic specifications that apply to any platform.  See
    http://www.codesourcery.com/cxx-abi/.

  libstdc++ ABI: There is no formal specification of this, it's just
    how the library is actually implemented.

  LSB: The Linux Standards Base specifies binary interfaces that allow
    compliant applications to run on a variety of conforming GNU/Linux
    distributions.  See http://www.linuxbase.org/FAQ.html.

  ABI conformance: compiler or library is implemented according to an
    ABI, or application is built using only tools that conform to a set
    of ABIs

  compatibility: two different compilers, or two versions of the same
    compiler, generate object files that can be linked together

  consistency: the same compiler does the same thing from one release to
    another

  interoperability: ability to link relocatable objects built by
    different compilers that conform to the same C++ ABI and that were
    built using header files for the same or compatible implementations
    of the C++ Standard Library