[RFA] make bootstrap-debug default, add broader -fcompare-debug testing options

[Alexandre Oliva <aoliva at redhat dot com>]

This patch implements richi's and RTH's suggestions, and it requires
review to some command-line flag processing changes.

It makes bootstrap-debug by default, building stage2 without -g and
stage3 with it, and using compare-debug to compare.

It also introduces bootstrap-debug-big and bootstrap-debug-lean to
perform -fcompare-debug-like testing.

-big outputs the final insn dumps that -fcompare-debug uses during
stage2 and stage3, so that the compare-debug script compares them.

-lean saves the disk space to hold all such dumps at the same time (it's
a lot) and uses -fcompare-debug proper, generating dumps that should be
equivalent to stage2's during stage3.

In theory, the former might be faster, but the extra disk space and the
fact that stage2 is compiled without optimization might actually make it
slower.  I haven't compared them myself.


Implementing -big required some ugly changes to gcc.c and toplev.c: we
want the final insn dump to be named after the compiler output, so that
compare-debug can easily find it, but adding -fdump-final-insns=$@.gkd
to BOOT_CFLAGS feels too risky.  So I tweaked the compiler driver to
accept -fdump-final-insns and turn that into what
-fdump-final-insns=$@.gkd would have expanded to (assuming -o $@ or -c
without -o).

Much of this patch has been in the VTA branch for a few days already.
This patch combines that recent patch with some other pieces of
documentation and build configurations that were still maintained in the
VTA branch, but that did not have any dependencies on VTA changes, or
that were changed so as to not have them.

Are the gcc.c and toplev.c changes ok for the trunk?

Is the change to the default bootstrap behavior, so that it's faster and
gets better -g/-g0 coverage, ok?

for  ChangeLog
from  Alexandre Oliva  <aoliva@redhat.com>

	* Makefile.tpl (BUILD_CONFIG): Default to bootstrap-debug.
	* Makefile.in: Rebuilt.

for  config/ChangeLog
from  Alexandre Oliva  <aoliva@redhat.com>

	* bootstrap-debug.mk: Add comments.
	* bootstrap-debug-big.mk: New.
	* bootstrap-debug-lean.mk: New.
	* bootstrap-debug-ckovw.mk: Add comments.
	* bootstrap-debug-lib.mk: Drop CFLAGS for stages.  Use -g0
	for TFLAGS in stage1.  Drop -fvar-tracking-assignments-toggle.

for  contrib/ChangeLog
from  Alexandre Oliva  <aoliva@redhat.com>

	* compare-debug: Look for .gkd files and compare them.

for  gcc/ChangeLog
from  Alexandre Oliva  <aoliva@redhat.com>

	* doc/invoke.texi (bootstrap-debug): More details.
	(bootstrap-debug-big, bootstrap-debug-lean): Document.
	(bootstrap-debug-lib): More details.
	(bootstrap-debug-ckovw): Update.
	* gcc.c (invoke_as): Call compare-debug-dump-opt for
	-fdump-final-insns too.
	(driver_self_specs): Map -fdump-final-insns to
	-fdump-final-insns=..
	(compare_debug_dump_opt_spec_function): Test for . argument and
	compute output name.  Compute temp output spec without flag name.
	* doc/invoke.texi (fdump-final-insns): Document argument-less use.
	* toplev.c (process_options): Don't open final insns dump file if
	we're not going to write to it.

Index: gcc/doc/install.texi
===================================================================
--- gcc/doc/install.texi.orig	2009-06-11 08:52:31.000000000 -0300
+++ gcc/doc/install.texi	2009-06-11 08:55:00.000000000 -0300
@@ -2040,8 +2040,53 @@ Removes any @option{-O}-started option f
 Analogous to @code{bootstrap-O1}.
 
 @item @samp{bootstrap-debug}
-Builds stage2 without debug information, and uses
-@file{contrib/compare-debug} to compare object files.
+Verifies that the compiler generates the same executable code, whether
+or not it is asked to emit debug information.  To this end, this option
+builds stage2 host programs without debug information, and uses
+@file{contrib/compare-debug} to compare them with the stripped stage3
+object files.  If @code{BOOT_CFLAGS} is overridden so as to not enable
+debug information, stage2 will have it, and stage3 won't.  This option
+is enabled by default when GCC bootstrapping is enabled: in addition to
+better test coverage, it makes default bootstraps faster and leaner.
+
+@item @samp{bootstrap-debug-big}
+In addition to the checking performed by @code{bootstrap-debug}, this
+option saves internal compiler dumps during stage2 and stage3 and
+compares them as well, which helps catch additional potential problems,
+but at a great cost in terms of disk space.
+
+@item @samp{bootstrap-debug-lean}
+This option saves disk space compared with @code{bootstrap-debug-big},
+but at the expense of some recompilation.  Instead of saving the dumps
+of stage2 and stage3 until the final compare, it uses
+@option{-fcompare-debug} to generate, compare and remove the dumps
+during stage3, repeating the compilation that already took place in
+stage2, whose dumps were not saved.
+
+@item @samp{bootstrap-debug-lib}
+This option tests executable code invariance over debug information
+generation on target libraries, just like @code{bootstrap-debug-lean}
+tests it on host programs.  It builds stage3 libraries with
+@option{-fcompare-debug}, and it can be used along with any of the
+@code{bootstrap-debug} options above.
+
+There aren't @code{-lean} or @code{-big} counterparts to this option
+because most libraries are only build in stage3, so bootstrap compares
+would not get significant coverage.  Moreover, the few libraries built
+in stage2 are used in stage3 host programs, so we wouldn't want to
+compile stage2 libraries with different options for comparison purposes.
+
+@item @samp{bootstrap-debug-ckovw}
+Arranges for error messages to be issued if the compiler built on any
+stage is run without the option @option{-fcompare-debug}.  This is
+useful to verify the full @option{-fcompare-debug} testing coverage.  It
+must be used along with @code{bootstrap-debug-lean} and
+@code{bootstrap-debug-lib}.
+
+@item @samp{bootstrap-time}
+Arranges for the run time of each program started by the GCC driver,
+built in any stage, to be logged to @file{time.log}, in the top level of
+the build tree.
 
 @end table
 
Index: config/bootstrap-debug.mk
===================================================================
--- config/bootstrap-debug.mk.orig	2009-06-11 08:52:31.000000000 -0300
+++ config/bootstrap-debug.mk	2009-06-11 08:55:00.000000000 -0300
@@ -1,2 +1,12 @@
-STAGE2_CFLAGS += -g0
+# This BUILD_CONFIG option builds checks that toggling debug
+# information generation doesn't affect the generated object code.
+
+# It is very lightweight: in addition to not performing any additional
+# compilation (unlike bootstrap-debug-lean), it actually speeds up
+# stage2, for no debug information is generated when compiling with
+# the unoptimized stage1.
+
+# For more thorough testing, see bootstrap-debug-lean.mk
+
+STAGE2_CFLAGS += -gtoggle
 do-compare = $(SHELL) $(srcdir)/contrib/compare-debug $$f1 $$f2
Index: config/bootstrap-debug-big.mk
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ config/bootstrap-debug-big.mk	2009-06-11 08:55:00.000000000 -0300
@@ -0,0 +1,8 @@
+# This BUILD_CONFIG option is a bit like bootstrap-debug-lean, but it
+# trades space for speed: instead of recompiling programs during
+# stage3, it generates dumps during stage2 and stage3, saving them all
+# until the final compare.
+
+STAGE2_CFLAGS += -gtoggle -fdump-final-insns
+STAGE3_CFLAGS += -fdump-final-insns
+do-compare = $(SHELL) $(srcdir)/contrib/compare-debug $$f1 $$f2
Index: config/bootstrap-debug-lean.mk
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ config/bootstrap-debug-lean.mk	2009-06-11 08:55:00.000000000 -0300
@@ -0,0 +1,11 @@
+# This BUILD_CONFIG option is a bit like bootstrap-debug, but in
+# addition to comparing stripped object files, it also compares
+# compiler internal state during stage3.
+
+# This makes it slower than bootstrap-debug, for there's additional
+# dumping and recompilation during stage3.  bootstrap-debug-big can
+# avoid the recompilation, if plenty of disk space is available.
+
+STAGE2_CFLAGS += -gtoggle -fcompare-debug=
+STAGE3_CFLAGS += -fcompare-debug
+do-compare = $(SHELL) $(srcdir)/contrib/compare-debug $$f1 $$f2
Index: config/bootstrap-time.mk
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ config/bootstrap-time.mk	2009-06-11 08:52:37.000000000 -0300
@@ -0,0 +1,2 @@
+BOOT_CFLAGS += -time=$(shell pwd)/time.log
+TFLAGS += -time=$(shell pwd)/time.log
Index: config/bootstrap-debug-ckovw.mk
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ config/bootstrap-debug-ckovw.mk	2009-06-11 08:55:00.000000000 -0300
@@ -0,0 +1,16 @@
+# This BUILD_CONFIG option is to be used along with
+# bootstrap-debug-lean and bootstrap-debug-lib in a full bootstrap, to
+# check that all host and target files are built with -fcompare-debug.
+
+# These arrange for a simple warning to be issued if -fcompare-debug
+# is not given.
+# BOOT_CFLAGS += -fcompare-debug="-w%n-fcompare-debug not overridden"
+# TFLAGS += -fcompare-debug="-w%n-fcompare-debug not overridden"
+
+# GCC_COMPARE_DEBUG="-w%n-fcompare-debug not overridden";
+
+FORCE_COMPARE_DEBUG = \
+	GCC_COMPARE_DEBUG=$${GCC_COMPARE_DEBUG--fcompare-debug-not-overridden}; \
+	export GCC_COMPARE_DEBUG;
+POSTSTAGE1_HOST_EXPORTS += $(FORCE_COMPARE_DEBUG)
+BASE_TARGET_EXPORTS += $(FORCE_COMPARE_DEBUG)
Index: config/bootstrap-debug-lib.mk
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ config/bootstrap-debug-lib.mk	2009-06-11 08:55:00.000000000 -0300
@@ -0,0 +1,12 @@
+# This BUILD_CONFIG option tests that target libraries built during
+# stage3 would have generated the same executable code if they were
+# compiled with -g0.
+
+# It uses -g0 rather than -gtoggle because -g is default on target
+# library builds, and toggling it where it's supposed to be disabled
+# breaks e.g. crtstuff on ppc.
+
+STAGE1_TFLAGS += -g0 -fcompare-debug=
+STAGE2_TFLAGS += -fcompare-debug=
+STAGE3_TFLAGS += -fcompare-debug=-g0
+do-compare = $(SHELL) $(srcdir)/contrib/compare-debug $$f1 $$f2
Index: Makefile.in
===================================================================
--- Makefile.in.orig	2009-06-10 09:54:49.000000000 -0300
+++ Makefile.in	2009-06-11 08:55:00.000000000 -0300
@@ -740,7 +740,7 @@ EXTRA_GCC_FLAGS = \
 GCC_FLAGS_TO_PASS = $(BASE_FLAGS_TO_PASS) $(EXTRA_HOST_FLAGS) $(EXTRA_GCC_FLAGS)
 
 @if gcc
-BUILD_CONFIG =
+BUILD_CONFIG = bootstrap-debug
 ifneq ($(BUILD_CONFIG),)
 include $(foreach CONFIG, $(BUILD_CONFIG), $(srcdir)/config/$(CONFIG).mk)
 endif
Index: Makefile.tpl
===================================================================
--- Makefile.tpl.orig	2009-06-05 04:00:57.000000000 -0300
+++ Makefile.tpl	2009-06-11 08:55:00.000000000 -0300
@@ -590,7 +590,7 @@ EXTRA_GCC_FLAGS = \
 GCC_FLAGS_TO_PASS = $(BASE_FLAGS_TO_PASS) $(EXTRA_HOST_FLAGS) $(EXTRA_GCC_FLAGS)
 
 @if gcc
-BUILD_CONFIG =
+BUILD_CONFIG = bootstrap-debug
 ifneq ($(BUILD_CONFIG),)
 include $(foreach CONFIG, $(BUILD_CONFIG), $(srcdir)/config/$(CONFIG).mk)
 endif
Index: contrib/compare-debug
===================================================================
--- contrib/compare-debug.orig	2009-06-05 04:01:37.000000000 -0300
+++ contrib/compare-debug	2009-06-11 08:55:00.000000000 -0300
@@ -149,4 +149,12 @@ $rm "$1.$suf1" "$2.$suf2"
 
 trap "exit $status; exit" 0 1 2 15
 
+if test -f "$1".gkd || test -f "$2".gkd; then
+  if cmp "$1".gkd "$2".gkd; then
+    :
+  else
+    status=$?
+  fi
+fi
+
 exit $status
Index: gcc/doc/invoke.texi
===================================================================
--- gcc/doc/invoke.texi.orig	2009-06-11 08:52:30.000000000 -0300
+++ gcc/doc/invoke.texi	2009-06-11 08:55:00.000000000 -0300
@@ -4395,9 +4395,12 @@ other options are processed, and it does
 many times it is given.  This is mainly intended to be used with
 @option{-fcompare-debug}.
 
-@item -fdump-final-insns=@var{file}
-@opindex fdump-final-insns=
-Dump the final internal representation (RTL) to @var{file}.
+@item -fdump-final-insns@r{[}=@var{file}@r{]}
+@opindex fdump-final-insns
+Dump the final internal representation (RTL) to @var{file}.  If the
+optional argument is omitted (or if @var{file} is @code{.}), the name
+of the dump file will be determined by appending @code{.gkd} to the
+compilation output file name.
 
 @item -fcompare-debug@r{[}=@var{opts}@r{]}
 @opindex fcompare-debug
Index: gcc/gcc.c
===================================================================
--- gcc/gcc.c.orig	2009-06-10 09:55:22.000000000 -0300
+++ gcc/gcc.c	2009-06-11 08:55:00.000000000 -0300
@@ -891,10 +891,10 @@ static const char *asm_options =
 
 static const char *invoke_as =
 #ifdef AS_NEEDS_DASH_FOR_PIPED_INPUT
-"%{fcompare-debug=*:%:compare-debug-dump-opt()}\
+"%{fcompare-debug=*|fdump-final-insns=*:%:compare-debug-dump-opt()}\
  %{!S:-o %|.s |\n as %(asm_options) %|.s %A }";
 #else
-"%{fcompare-debug=*:%:compare-debug-dump-opt()}\
+"%{fcompare-debug=*|fdump-final-insns=*:%:compare-debug-dump-opt()}\
  %{!S:-o %|.s |\n as %(asm_options) %m.s %A }";
 #endif
 
@@ -926,6 +926,7 @@ static const char *const multilib_defaul
 #endif
 
 static const char *const driver_self_specs[] = {
+  "%{fdump-final-insns:-fdump-final-insns=.} %<fdump-final-insns",
   DRIVER_SELF_SPECS, GOMP_SELF_SPECS
 };
 
@@ -8675,29 +8676,38 @@ compare_debug_dump_opt_spec_function (in
   if (arg != 0)
     fatal ("too many arguments to %%:compare-debug-dump-opt");
 
-  if (!compare_debug)
-    return NULL;
-
   do_spec_2 ("%{fdump-final-insns=*:%*}");
   do_spec_1 (" ", 0, NULL);
 
-  if (argbuf_index > 0)
+  if (argbuf_index > 0 && strcmp (argv[argbuf_index - 1], "."))
     {
+      if (!compare_debug)
+	return NULL;
+
       name = xstrdup (argv[argbuf_index - 1]);
       ret = NULL;
     }
   else
     {
-#define OPT "-fdump-final-insns="
-      ret = "-fdump-final-insns=%g.gkd";
+      const char *ext = NULL;
+
+      if (argbuf_index > 0)
+	{
+	  do_spec_2 ("%{o*:%*}%{!o:%{!S:%b%O}%{S:%b.s}}");
+	  ext = ".gkd";
+	}
+      else if (!compare_debug)
+	return NULL;
+      else
+	do_spec_2 ("%g.gkd");
 
-      do_spec_2 (ret + sizeof (OPT) - 1);
       do_spec_1 (" ", 0, NULL);
-#undef OPT
 
       gcc_assert (argbuf_index > 0);
 
-      name = xstrdup (argbuf[argbuf_index - 1]);
+      name = concat (argbuf[argbuf_index - 1], ext, NULL);
+
+      ret = concat ("-fdump-final-insns=", name, NULL);
     }
 
   which = compare_debug < 0;
Index: gcc/toplev.c
===================================================================
--- gcc/toplev.c.orig	2009-06-11 08:52:30.000000000 -0300
+++ gcc/toplev.c	2009-06-11 08:55:00.000000000 -0300
@@ -1878,7 +1878,7 @@ process_options (void)
 	debug_info_level = DINFO_LEVEL_NONE;
     }
 
-  if (flag_dump_final_insns)
+  if (flag_dump_final_insns && !flag_syntax_only && !no_backend)
     {
       FILE *final_output = fopen (flag_dump_final_insns, "w");
       if (!final_output)

-- 
Alexandre Oliva, freedom fighter    http://FSFLA.org/~lxoliva/
You must be the change you wish to see in the world. -- Gandhi
Be Free! -- http://FSFLA.org/   FSF Latin America board member
Free Software Evangelist      Red Hat Brazil Compiler Engineer