Re: Testcase minimization with Delta

[Zack Weinberg <zack at codesourcery dot com>]

Daniel Berlin <dberlin@dberlin.org> writes:

> I found a neat program that can do testcase minimization for GCC.
> It's a bit slow in it's current form, but this is from execing gcc, AFAICT. If 
> someone rewrote it in C (not difficult, there are python and perl 
> implementations of the algorithm it uses, and it's fairly straightforward)  
> so that it could be linked with gcc itself, we could have an automatic 
> test case minimizer program.

I tried this out on a messy test case (spline.ii from PR 8511).  It
did a nice job of reducing the original test case from 32,000 lines
down to 5,000 -- but getting it smaller than that rapidly hit
diminishing returns.  In the end it hit a wall at about 1000 lines; I
was able to take that and cut it down by hand to 44 lines.  I think
much of this is because it only knows how to do deletions, not
substitutions or reorderings; but it was also remarkably bad at
catching huge chunks that were unnecessary (deletions that were
obvious to a human).  This could no doubt be improved if it had some
knowledge of the structure of a C++ program.

It is also painfully slow, and I don't think it's fair to blame all of
that on the time spent execing the compiler.  The algorithm appears
rather more brute force it needs to be - it does things like iterate
over the file trying each possible one-line removal, for instance.
And I don't think it's practical, with the present state of the
compiler, to attempt to run it over and over again without starting a
new process for each attempt.

Joe Bob says "interesting, but not ready for prime time."

zw