This is the mail archive of the
gcc-patches@gcc.gnu.org
mailing list for the GCC project.
Re: Bug reduction instructions, draft 2
- From: Wolfgang Bangerth <bangerth at ticam dot utexas dot edu>
- To: Gerald Pfeifer <pfeifer at dbai dot tuwien dot ac dot at>
- Cc: gcc-patches at gcc dot gnu dot org, Janis Johnson <janis187 at us dot ibm dot com>
- Date: Tue, 14 Jan 2003 10:01:18 -0600 (CST)
- Subject: Re: Bug reduction instructions, draft 2
> (b) I may be biased (as I am currently" compressing" a paper for
> submission to conference) but can we compress this somehow?
How we scientists hate these page limits -- we should start to write our
papers in the form of novels...
But it's all true what you say, so I tried to squeeze it a bit, and also
fixed a couple of things Volker noted. If someone wants to squeeze harder,
please feel free. Appended is draft 2.
W.
--- empty Fri Dec 13 17:01:24 2002
+++ minimize-howto.html Fri Jan 10 12:23:31 2003
@@ -0,0 +1,208 @@
+<html>
+
+<head>
+<title>How to Minimize Test Cases for Bugs</title>
+</head>
+
+<body>
+
+<h1>How to Minimize Test Cases for Bugs</h1>
+
+<p>In order for a GCC developer to fix a bug, the bug must be
+reproducible by means of a self-contained test case. Our
+<a href="http://gcc.gnu.org/bugs.html";>bug reporting instructions</a>
+ask that a bug report include the preprocessed version of the file that
+triggers the bug. Often this file is very large; there are several
+reasons for making it as small as possible:</p>
+
+<ul>
+ <li>Removing/reducing the amount of proprietary source may be required
+ to allow publication.</li>
+
+ <li>Fewer dependencies on specific target system features, such as the
+ content of system headers, make the test more easily reproducible.</li>
+
+ <li>Smaller test cases make debugging easier.</li>
+
+ <li>GCC developers are more likely to tackle bug reports that have
+ small, portable test cases.</li>
+
+ <li>Minimized test cases can be used for GCC's regression test suites.</li>
+</ul>
+
+<p>Experience shows that most testcases can be reduced to fewer than
+30 lines, and often fewer than ten lines.</p>
+
+<h2>Direct approach</h2>
+
+<p>For a bogus error or warning message from the compiler or for an
+internal compiler error (ICE) that reports a line number from the source
+file being compiled, it's often possible to write a very small test
+case based on the code at the location reported in the messages.
+Such a test case doesn't need to include any header files; if it must
+call a library function it can specify the declaration for that
+function directly (e.g. <code>extern "C" int printf(const char *, ...);
+</code>).</p>
+
+<p>Likewise, suspected bugs in a GCC runtime
+library can often be demonstrated with a small test case that calls
+that function, either directly or indirectly. Such a test then of course
+needs to include library header files.</p>
+
+<p>Always try this direct approach before resorting to the brute-force
+method of minimizing a large test case.</p>
+
+<h2>Brute force approach</h2>
+
+<p>The brute force approach gets easier with experience. The first
+time it can take an hour or two, but experienced people can often make
+it in 20 minutes or so per test case.</p>
+
+<p>After each attempt to shorten the test case, check whether the bug
+still is evident. If not, back up to the previous version that
+demonstrated the bug, and try something else. For this, either use the undo
+function of your editor, or use frequent backup files; alternatively, you
+can use <code>#if 0</code> and <code>#endif</code> to remove parts of
+code.</p>
+
+<p>If you're reporting the bug, you've got the choice of starting
+with either the original source files or with a preprocessed file.
+Both methods are described below.</p>
+
+<h3>Stripping the original code</h3>
+
+<p>Copy the original source file, plus header files that you might need
+to modify, to a separate location where you can duplicate the failure.
+Each of the following steps is an attempt to minimize the test case; some will
+be successful, some will not. If something is unsuccessful, undo it
+and move on to a different step.</p>
+
+<ul>
+ <li>Delete all functions that come after the function in which the
+ bug happens.
+
+ <ul>
+ <li>If the bug happens within an inline function, make sure that
+ the function is actually used (write a dummy function that calls it)
+ or remove the <code>inline</code> directive.</li>
+
+ <li>If the bug happens in a template function, it must be
+ instantiated. You can do this with an explicit instantiation.</li>
+ </ul></li>
+
+ <li>Strip as much code as possible from the function in which the
+ problem happens. It's not important if the remaining code doesn't
+ make any sense, it only needs to trigger the bug.
+
+ <ul>
+ <li>Remove definitions of variables that are no longer used.</li>
+ <li>Replace variables of user-defined types (structs, classes) with
+ builtin types (<code>int</code>, <code>double</code>).</li>
+ <li>Replace typedefs with primitive types, or with the types they
+ reference.</li>
+ <li>In C++, remove I/O statements. This will allow you to remove
+ inclusion of <code>iostream</code> headers, saving thousands of lines in
+ the preprocessed version of the source file. If you need output to
+ demonstrate the bug, use <code>printf</code>, and specify its
+ declaration directly: <code>extern "C" int printf(const char *, ...);
+ </code>.</li>
+ </ul></li>
+
+ <li>Delete functions that come before the function that causes
+ the bug. If they are referenced later, replace their definitions with
+ declarations.</li>
+
+ <li>Rearrange the order of <code>#include</code> directives so that
+ system headers come first; this can make the rest of the process
+ simpler.</li>
+
+ <li>Remove <code>#include</code> directives, or replace them with
+ the code that is needed from them in order to get the file to
+ continue to compile; this will help when you start cutting down the
+ preprocessed file.</li>
+
+ <li>Try to remove C++ template arguments from functions and classes.
+ Don't try this too early; it involves changing many places if there
+ are still too many references to the template.</li>
+
+ <li>Iterate through the following:
+ <ul>
+ <li>Remove class and struct members that are no longer referenced.</li>
+
+ <li>Remove base classes or use the base class instead of the derived one.
+ Make virtual functions non-virtual.</li>
+
+ <li>Remove definitions for classes, structs, and other data types
+ that are no longer used.</li>
+
+ <li>Remove includes of header files that are no longer needed.</li>
+ </ul>
+ </li>
+
+</ul>
+
+<p>The file in which the problem happens will now usually be down to
+20 or 30 lines, plus the necessary include directives.
+Repeat these steps with the header files of your project that are
+still included in the file that shows the problem.
+To reduce the number of files you are working on, you can directly
+include the content of the header file into the source file you are
+working on.</p>
+
+<p>Prepare for the next step by running the compiler with
+<code>-save-temps</code> to get the preprocessed source file.</p>
+
+<h2>Stripping preprocessed sources</h2>
+
+<p>The preprocessed file contains lines starting with <code>#</code>
+that tell the compiler the original file and line number of each line
+of code. You'll want to get rid of these lines so the compiler will
+report the location within the preprecessed file. For a preprocessed
+file called <code>bug.ii</code>, any of the following will do this:</p>
+
+<ul>
+ <li><code>perl -pi -e 's/^#.*\n//g;' bug.ii</code></li>
+ <li><code>sed 's/^#/d' bug.ii > bug2.ii</code></li>
+ <li>within <code>vi</code>: <code>:g/^#/d</code></li>
+</ul>
+
+<p>The preprocessed sources will now consist largely of header files.
+Follow the same steps as for stripping the original code, with these
+additional techniques.</p>
+
+<ul>
+ <li>There will be large stretches of typedefs from system headers
+ and, for C++, things enclosed in namespace <code>std</code>. Delete
+ large chunks working from the bottom up, e.g. whole <code>extern "C"</code>
+ blocks, or blocks enclosed in <code>namespace XXX {...}</code>.</li>
+
+ <li>If you get a new compiler error due to a missing definition for
+ a function or type that you deleted, replace the whole deleted chunk
+ with only the missing declaration, or modify the code to no longer
+ use it.</li>
+</ul>
+
+<p>At this stage, you will be able to delete chunks of hundreds or even
+thousands of lines at a time, and you will quickly be able to reduce the
+preprocessed sources to something small.</p>
+
+</body>
+</html>
-------------------------------------------------------------------------
Wolfgang Bangerth email: bangerth@ticam.utexas.edu
www: http://www.ticam.utexas.edu/~bangerth/