re: Spurious testsuite failures

[Dan Kegel <dank at kegel dot com>]

Christian Ehrhardt wrote:
> I'm getting lots of testsuite failures on sparc-sun-solaris. However,
> these failures don't seem to be consistent over testsuit runs. I ran
> the g++ testsuite twice in a row (with the same compiler!) once with 119
> failures and a second time with only 50 failures. Closer inspection shows
> that only 8 failures are common between the two consecutive runs.
>
> > From looking at the testsuite log it seems that some error messages don't
> make it to the log. All of the suprious failures are either
> "test for error/warning" or "test for excess errors" The latter happens
> if previous error message were missed and the compiler still exits with
> a nonzero exit status.
>
> Has anyone seen anything like this before? DejaGNU-Version is 1.4.1.

Yes.  You're running into http://gcc.gnu.org/PR12096
It's a bug in expect.  I have a copy of the fix at
http://kegel.com/crosstool/current/patches/expect-5.39/pr12096.patch

For what it's worth, here's a script I wrote that creates a good
copy of expect/tcl/dejagnu from source (assuming you have that
patch in the right place).  Untested.  (I also have a patch for dejagnu
and tcl each, but you probably don't need 'em.)
- Dan

#!/bin/sh
# Compile local copies of expect, dejagnu, and distcc
# for use by crosstool

set -ex

abort() {
        echo $@
        exec false
}

test -z "${RESULT_TOP}"       && abort "Please set RESULT_TOP to the top of the crosstool tree"
test -z "${TARBALLS_DIR}"     && abort "Please set TARBALLS_DIR to the directory to download tarballs to."

#----- Tcl -----
TCL=tcl8.4.5
TCLDIR=tcl8.4.5
# !$1#$@!& sourceforge.net doesn't support wget -c ?!
if test ! -f $TARBALLS_DIR/$TCL-src.tar.gz ; then
    wget -P $TARBALLS_DIR http://unc.dl.sourceforge.net/sourceforge/tcl/$TCL-src.tar.gz
fi
rm -rf $TCLDIR
tar -xzvf $TARBALLS_DIR/$TCL-src.tar.gz
cd $TCLDIR
for a in ../../patches/$TCL/*.patch; do
    if test -f $a; then
        patch -g0 -p1 < $a
    fi
done
cd unix
./configure --prefix=$PREFIX
make
make install
cd ../..

#----- Expect -----
EXPECT=expect-5.39.0
EXPECTDIR=expect-5.39
wget -P $TARBALLS_DIR -c http://expect.nist.gov/old/$EXPECT.tar.gz
rm -rf $EXPECTDIR
tar -xzvf $TARBALLS_DIR/$EXPECT.tar.gz
cd $EXPECTDIR
for a in ../../patches/$EXPECT/*.patch; do
    if test -f $a; then
        patch -g0 -p1 < $a
    fi
done
./configure --prefix=$PREFIX
make
make install
cd ..

#----- Dejagnu -----
DEJAGNU=dejagnu-1.4.3
wget -P $TARBALLS_DIR -c http://mirrors.usc.edu/pub/gnu/dejagnu/$DEJAGNU.tar.gz
rm -rf $DEJAGNU
tar -xzvf $TARBALLS_DIR/$DEJAGNU.tar.gz
cd $DEJAGNU
for a in ../../patches/$DEJAGNU/*.patch; do
    if test -f $a; then
        patch -g0 -p1 < $a
    fi
done
./configure --prefix=$PREFIX
make
make install
cd ..

--
My technical stuff: http://kegel.com
My politics: see http://www.misleader.org for examples of why I'm for regime change