Re: Testcase minimization with Delta

[Daniel Berlin <dberlin at dberlin dot org>]

On Sat, 16 Nov 2002, Zack Weinberg wrote:

> Daniel Berlin <dberlin@dberlin.org> writes:
> 
> > I found a neat program that can do testcase minimization for GCC.
> > It's a bit slow in it's current form, but this is from execing gcc, AFAICT. If 
> > someone rewrote it in C (not difficult, there are python and perl 
> > implementations of the algorithm it uses, and it's fairly straightforward)  
> > so that it could be linked with gcc itself, we could have an automatic 
> > test case minimizer program.
> 
> I tried this out on a messy test case (spline.ii from PR 8511).  It
> did a nice job of reducing the original test case from 32,000 lines
> down to 5,000 -- but getting it smaller than that rapidly hit
> diminishing returns.  In the end it hit a wall at about 1000 lines; I
> was able to take that and cut it down by hand to 44 lines.  I think
> much of this is because it only knows how to do deletions, not
> substitutions or reorderings; but it was also remarkably bad at
> catching huge chunks that were unnecessary (deletions that were
> obvious to a human).  This could no doubt be improved if it had some
> knowledge of the structure of a C++ program.

You didn't run topformflat on the program, did you?

> 
> It is also painfully slow, and I don't think it's fair to blame all of
> that on the time spent execing the compiler.  The algorithm appears
> rather more brute force it needs to be - it does things like iterate
> over the file trying each possible one-line removal, for instance.
> And I don't think it's practical, with the present state of the
> compiler, to attempt to run it over and over again without starting a
> new process for each attempt.
> 
> Joe Bob says "interesting, but not ready for prime time."
> 
> zw
>