A Guide to Testcase Reduction

Our bug reporting instructions ask that a bug report include the preprocessed version of the file that triggers the bug. There are several methods to minimise a testcase.

This page ought to serve as an introduction to automatic testcase reduction using the Delta or C-Reduce tools.

For Fortran there is a patched version of Delta which takes subroutine/do/if boundaries into account.

Simple ICE reduction

  > cat testcase.i | grep -v '^# .*$' | grep -v '^[[:space:]]*$' > testcase.x
  > mv testcase.x testcase.i

  #!/bin/sh
  gcc -c -O -Wfatal-errors $1 2>&1 | grep 'internal compiler error: in typeck.c:2534'
  if ! test $? = 0; then
   exit 1
  fi
  exit 0

  > ~/bin/delta -test=check.sh -suffix=.i -cp_minimal=testcase-min.i testcase.i

Using topformflat

  > ~/bin/topformflat 0 x < testcase.i > testcase.0x.i

Using multidelta

All the above can be simplified by using the multidelta tool that comes with the Delta distribution. The only differences is that the script should be able to be called without parameters. In the above example, the new script would be:

  #!/bin/bash
  TESTCASE=${1:-testcase.i}
  gcc -c -O -Wfatal-errors -w $TESTCASE 2>&1 | grep -q 'internal compiler error: in typeck.c:2534'
  if ! test $? = 0; then
   exit 1
  fi
  exit 0

The fastest is this script, the fastest the reduction will be:

Then, you may run multidelta with the command:

multidelta -level=2 ./check.sh testcase.i &> /dev/null

Where -level= is the level of topformflat (see above). Check the progress in the files log and multidelta.log. Multidelta modifies the input file, but it creates a backup .bak file). The last successful reduction is always stored in testcase.i.ok.

Using C-Reduce

Instead of using delta or multidelta it is also possible to run C-Reduce on your testcase. This reducer specifically targets C and C++ code and makes coordinated changes across the whole program: removing an array dimension, removing a function argument, reordering function calls, etc. It will first automatically run a few passes that are identical to multidelta, so there is no need to run delta or mulitdelta first. Creduce runs in parallel by default if multiple cores are available.

C-Reduce can be found here: http://embed.cs.utah.edu/creduce/

You may run C-Reduce with the same script that multidelta uses above:

creduce check.sh testcase.i

Reducing "works with -O, doesn't work with -O2" type bugs

  #!/bin/sh
  gcc -o works $1 -O -Wfatal-errors
  if ! test "$?" = "0"; then
    exit 1
  fi
  ( ulimit -t 10; ./works )
  if ! test "$?" = "0"; then
    exit 1
  fi
  gcc -o fails $1 -O2 -Wfatal-errors
  if ! test "$?" = "0"; then
    exit 1
  fi
  ( ulimit -t 10; ./fails )
  if test "$?" = "0"; then
    exit 1
  fi
  exit 0

Reducing LTO bugs

  #!/bin/sh

  /path/to/lto1 -o /dev/null @$1 rest-of-your-options 2>&1 | grep '...the ICE...'
  if ! test $? = 0; then
    exit 1
  fi

  gcc -r -nostdlib preprocessed-inputs rest-of-your-options

Further hints

 #!/bin/sh
 cat $1 ../../tail.i > x.i
 gcc -S x.i -Wfatal-errors
 if ! test "$?" = "0"; then
   exit 1
 fi
 exit 0

 1,/^_Z8cpp_testRK5ArrayILi2Ed10BrickViewUES3_:/d
 /\.size/,$d
 ,w

 #!/bin/sh
 cat $1 ../../tail.i > x.i
 gcc -S x.i -Wfatal-errors
 if ! test "$?" = "0"; then
   exit 1
 fi
 ed x.s < ../../testcase.ed
 diff -u ../../asm.s x.s
 if ! test "$?" = "0"; then
   exit 1
 fi
 exit 0

None: A_guide_to_testcase_reduction (last edited 2014-11-21 14:16:38 by TobiasBurnus)