[tree-ssa] Writing testcases for optimizations

[Chris Lattner <sabre at nondot dot org>]

It occurred to me that this might be interesting for other tree-ssa
people.

-Chris

-- 
http://llvm.cs.uiuc.edu/
http://www.nondot.org/~sabre/Projects/

---------- Forwarded message ----------
Date: Wed, 17 Dec 2003 23:29:48 -0600 (CST)
From: Chris Lattner <sabre@nondot.org>
To: law@redhat.com
Cc: Pop Sbastian <pop@gauvain.u-strasbg.fr>, gcc-patches@gcc.gnu.org
Subject: Re: [tree-ssa, LNO] Analysis of scalar evolutions and data dependences


Jeff Law wrote:
> As you get ideas, do feel free to bounce them off me -- I don't need it
> immediately, but I'm on the lookout for a good way to do testing of
> code motions.

In LLVM, we write all of our analysis/optimizer regression tests as
fragments of .ll files (usually a function or two) along with a command to
run the test.  Here's an example:

; RUN: llvm-as < %s | opt -licm | llvm-dis | grep -C1 volatile | grep Loop:
%X = global int 7
void %testfunc(int %i) {
        br label %Loop

Loop:
        %x = volatile load int* %X  ; Should not promote this to a register
        %x2 = add int %x, 1
        store int %x2, int* %X
        br bool true, label %Out, label %Loop

Out:
        ret void
}

Our QMTest harness turns the RUN: line into a bourne shell script,
transforming the %s into the filename, then executes the script.  In this
case, we assemble the testcase, run it through the LICM pass, disassemble
it to text, and check to make sure that the volatile load is still in the
loop.  If not, it will fail. Also, since the test isn't run through a lot
of extraneous transformations, strange temporaries and other problems
can't interfere with the tests, making them robust.  In this case, for
example, we don't have to worry about constant propagation turning the
Loop into straightline code, which allows the test to be simplified
dramatically.

The nice thing about this structure is that we know _exactly_ what we are
testing (no prior optimizations can mess up the input to the pass we are
testing), and that we can write extremely simple tests to check to see if
the test is successful or not.  The other big advantage of the LLVM system
is that we have an automated tool which can narrow down a miscompilation
or optimizer crash to a nice small testcase like the above, by treating
all of the optimizations like black boxes.  This is described in more
detail here: http://llvm.cs.uiuc.edu/docs/CommandGuide/bugpoint.html

To implement a system like this, you need a way to represent the program
being compiled _it its entirety_ in a textual format, and a modular
optimizer.  I believe that this could be added to tree-ssa without too
much of a problem if the desire was there.

-Chris

-- 
http://llvm.cs.uiuc.edu/
http://www.nondot.org/~sabre/Projects/