Created attachment 22058 [details]
preprocessed source for libcpp/lex.c

The attached preprocessed source, lex.i, for libcpp/lex.c provides an example of the code which triggers the bootstrap-lto failures on darwin....

/sw/src/fink.build/gcc46-4.6.0-1000/darwin_objdir/./prev-gcc/xgcc -B/sw/src/fink.build/gcc46-4.6.0-1000/darwin_objdir/./prev-gcc/ -B/sw/lib/gcc4.6/x86_64-apple-darwin10.5.0/bin/ -B/sw/lib/gcc4.6/x86_64-apple-darwin10.5.0/bin/ -B/sw/lib/gcc4.6/x86_64-apple-darwin10.5.0/lib/ -isystem /sw/lib/gcc4.6/x86_64-apple-darwin10.5.0/include -isystem /sw/lib/gcc4.6/x86_64-apple-darwin10.5.0/sys-include -I../../gcc-4.6-20101014/libcpp -I. -I../../gcc-4.6-20101014/libcpp/../include -I../../gcc-4.6-20101014/libcpp/include -I/sw/include -g -O2 -W -Wall -Wwrite-strings -Wmissing-format-attribute -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition -Wc++-compat -pedantic -Wno-long-long -Werror -I../../gcc-4.6-20101014/libcpp -I. -I../../gcc-4.6-20101014/libcpp/../include -I../../gcc-4.6-20101014/libcpp/include -I/sw/include -flto -c -o lex.o -MT lex.o -MMD -MP -MF .deps/lex.Tpo lex.i
../../gcc-4.6-20101014/libcpp/lex.c:2838:1: sorry, unimplemented: gimple bytecode streams do not support the optimization attribute

The same error appears when -fwhopr or -fwhopr=jobserver is used instead of -flto.

the optimize attribute isn't used in the preprocessed file but only the
target attribute which is supported.  Thus, "worksforme".

(In reply to comment #2)
> the optimize attribute isn't used in the preprocessed file but only the
> target attribute which is supported.  Thus, "worksforme".

Using a stock build of gcc trunk on x86_64-apple-darwin10, built with...

Using built-in specs.
COLLECT_GCC=gcc-4
COLLECT_LTO_WRAPPER=/sw/lib/gcc4.6/libexec/gcc/x86_64-apple-darwin10.5.0/4.6.0/lto-wrapper
Target: x86_64-apple-darwin10.5.0
Configured with: ../gcc-4.6-20101014/configure --prefix=/sw --prefix=/sw/lib/gcc4.6 --mandir=/sw/share/man --infodir=/sw/lib/gcc4.6/info --enable-languages=c,c++,fortran,objc,obj-c++,java,lto --with-gmp=/sw --with-libiconv-prefix=/sw --with-ppl=/sw --with-cloog=/sw --with-mpc=/sw --with-system-zlib --x-includes=/usr/X11R6/include --x-libraries=/usr/X11R6/lib --program-suffix=-fsf-4.6 --enable-checking=yes
Thread model: posix
gcc version 4.6.0 20101015 (experimental) (GCC) 

the error can be reproduced out of tree with the lex.i file as...

gcc-4 -g -O2 -W -Wall -Wwrite-strings -Wmissing-format-attribute -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition -Wc++-compat -pedantic -Wno-long-long -Werror -flto -c -o lex.o -MT ex.o -MMD -MP -MF .deps/lex.Tpo lex.i
../../gcc-4.6-20101014/libcpp/lex.c:2838:1: sorry, unimplemented: gimple bytecode streams do not support the optimization attribute

It would be interesting to know if COFF's lto support has the same issues (since they share the same backend design for lto support).

(In reply to comment #2)
> the optimize attribute isn't used in the preprocessed file but only the
> target attribute which is supported.  Thus, "worksforme".

Target attributes must be implying optimisation attributes.  Bug depends also on --with-{arch,tune,fpmath} settings.  I'll try and reproduce it on x86_64-linux, it should be possible if I choose the right settings - will reply again later.

(In reply to comment #3)

> It would be interesting to know if COFF's lto support has the same issues
> (since they share the same backend design for lto support).

No, it's not related to the backend lto support at all; the sorry comes from the lto output streamer.  It does depend on the backend's attribute handling though.

(See also PR41201, vaguely related.)

> Target attributes must be implying optimisation attributes.  Bug depends also
> on --with-{arch,tune,fpmath} settings.  I'll try and reproduce it on
> x86_64-linux, it should be possible if I choose the right settings - will reply
> again later.

Well, we probably should just go ahead and implement streaming of optimizaion attribute.
But still I don't see how this implicatoin happens.

Honza

Well, so the problem is:
static bool
ix86_valid_target_attribute_p (tree fndecl,
                               tree ARG_UNUSED (name),
                               tree args,
                               int ARG_UNUSED (flags))
{
...
  new_optimize = build_optimization_node ();
...
      if (old_optimize != new_optimize)
        DECL_FUNCTION_SPECIFIC_OPTIMIZATION (fndecl) = new_optimize;

for some reason the optimization flags are believed to not match despite the fact only target attribute is used.  The difference is:

- <optimization_node 0x143209100
+ <optimization_node 0x100fda500
     align_functions (0x10)
     align_jumps (0x10)
     align_labels (0x1)
@@ -95,5 +96,6 @@
     flag_unwind_tables (0x1)
     flag_var_tracking (0x1)
     flag_var_tracking_assignments (0x1)
+    flag_var_tracking_uninit (0x1)
     flag_vect_cost_model (0x1)
 >

so it seems that re-running process_options on darwin somehow leads to this change.  Adding Joseph to CC.

Still streaming of optimization attribute is thing we should do.

On Mon, 10 Jan 2011, hubicka at gcc dot gnu.org wrote:

> so it seems that re-running process_options on darwin somehow leads to this
> change.  Adding Joseph to CC.

How exactly is process_options rerun?  It's a static function in toplev.c, 
called once from another static function in toplev.c....  I guess maybe 
you mean decode_options.  In that case, maybe you need to implement 
TARGET_OVERRIDE_OPTIONS_AFTER_CHANGE to set at least some of the variables 
set by darwin_override_options.

I guess it is decode options.  But I wonder how we manage to play this ping-pong game.  I see darwin.c setting the uninit:

  if (flag_var_tracking
      && (generating_for_darwin_version >= 9)
      && debug_info_level >= DINFO_LEVEL_NORMAL
      && debug_hooks->var_location != do_nothing_debug_hooks.var_location)
    flag_var_tracking_uninit = 1;

and toplev.c clearing it:

  /* We know which debug output will be used so we can set flag_var_tracking
     and flag_var_tracking_uninit if the user has not specified them.  */
  if (debug_info_level < DINFO_LEVEL_NORMAL
      || debug_hooks->var_location == do_nothing_debug_hooks.var_location)
    { 
      if (flag_var_tracking == 1
          || flag_var_tracking_uninit == 1)
        { 
          if (debug_info_level < DINFO_LEVEL_NORMAL)
            warning (0, "variable tracking requested, but useless unless "
                     "producing debug info");
          else
            warning (0, "variable tracking requested, but not supported "
                     "by this debug format");
        }
      flag_var_tracking = 0;
      flag_var_tracking_uninit = 0;
    }

First time we go
process_options->darwin_override_options and we end up with flag disabled (can't verify as I am not big friend with Jack's gdb)
next time we go
#0  0x00000001009edb40 in darwin_override_options ()
#1  0x000000010099b120 in ix86_option_override_internal ()
#2  0x000000010099f15b in ix86_valid_target_attribute_tree ()
#3  0x000000010099f25b in ix86_valid_target_attribute_p ()
#4  0x000000010007ac04 in handle_target_attribute ()
#5  0x00000001000024ca in decl_attributes ()
this time we bypass process_options.

I don't see how those two conditionals can both match.


I also wonder why is darwin enabling the flag? 
It seems to be off at all configurations except for darwin even if looks like generic var tracking feature.  -fvar-tracking-uninit is also undocumented.

OK, first time the flag is not set because 
debug_hooks->var_location == do_nothing_debug_hooks.var_location
second time it is no longer true

Seems we need to ininitialize the debug hook earlier or use different test when processing options.

Honza

Hi,
this should fix the issue.
Index: config/darwin.c
===================================================================
--- config/darwin.c     (revision 168632)
+++ config/darwin.c     (working copy)
@@ -2618,7 +2618,7 @@
   if (flag_var_tracking
       && (generating_for_darwin_version >= 9)
       && debug_info_level >= DINFO_LEVEL_NORMAL
-      && debug_hooks->var_location != do_nothing_debug_hooks.var_location)
+      && write_symbols == DWARF2_DEBUG)
     flag_var_tracking_uninit = 1;
 
   if (MACHO_DYNAMIC_NO_PIC_P)

(In reply to comment #10)

While this patch allows the lto-bootstrap below to complete on x86_64-apple-darwin10...

../gcc/configure --enable-checking=release --prefix=/Users/howarth/dist --with-gmp=/sw --with-ppl=/sw --with-cloog=/sw --with-mpc=/sw --with-libiconv-prefix=/sw --with-system-zlib --x-includes=/usr/X11R6/include --x-libraries=/usr/X11R6/lib --with-build-config=bootstrap-lto --enable-stage1-languages=c,lto --enable-languages=c,lto

it breaks the standard bootstrap of...

../gcc/configure --enable-checking=release --prefix=/Users/howarth/dist --with-gmp=/sw --with-ppl=/sw --with-cloog=/sw --with-mpc=/sw --with-libiconv-prefix=/sw --with-system-zlib --x-includes=/usr/X11R6/include --x-libraries=/usr/X11R6/lib --enable-languages=c,lto

with the error...

/Users/howarth/darwin_objdir/./prev-gcc/xgcc -B/Users/howarth/darwin_objdir/./prev-gcc/ -B/Users/howarth/dist/x86_64-apple-darwin10.6.0/bin/ -B/Users/howarth/dist/x86_64-apple-darwin10.6.0/bin/ -B/Users/howarth/dist/x86_64-apple-darwin10.6.0/lib/ -isystem /Users/howarth/dist/x86_64-apple-darwin10.6.0/include -isystem /Users/howarth/dist/x86_64-apple-darwin10.6.0/sys-include     -I../../gcc/libcpp -I. -I../../gcc/libcpp/../include -I../../gcc/libcpp/include -I/sw/include -g -O2 -mdynamic-no-pic -gtoggle -W -Wall -Wwrite-strings -Wmissing-format-attribute -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition -Wc++-compat -pedantic -Wno-long-long -Werror -I../../gcc/libcpp -I. -I../../gcc/libcpp/../include -I../../gcc/libcpp/include -I/sw/include -c -o charset.o -MT charset.o -MMD -MP -MF .deps/charset.Tpo ../../gcc/libcpp/charset.c
../../gcc/libcpp/charset.c:1:0: error: variable tracking requested, but useless unless producing debug info [-Werror]
cc1: all warnings being treated as errors

make[3]: *** [charset.o] Error 1
make[2]: *** [all-stage2-libcpp] Error 2
make[1]: *** [stage2-bubble] Error 2
make: *** [all] Error 2

In case it helps, the code being patched was introduced with r126927 and was proposed in...

http://gcc.gnu.org/ml/gcc-patches/2007-07/msg01799.html

A slight correction. There appears to be no actual gcc-patches posting for r126927 (as it is an add-on fix to r126900)

2007-07-25  Daniel Berlin  <dberlin@dberlin.org>

	* config/darwin.c (darwin_override_options): Additional fix for
	debug info formats that don't support var tracking.

> /Users/howarth/darwin_objdir/./prev-gcc/xgcc
> -B/Users/howarth/darwin_objdir/./prev-gcc/
> -B/Users/howarth/dist/x86_64-apple-darwin10.6.0/bin/
> -B/Users/howarth/dist/x86_64-apple-darwin10.6.0/bin/
> -B/Users/howarth/dist/x86_64-apple-darwin10.6.0/lib/ -isystem
> /Users/howarth/dist/x86_64-apple-darwin10.6.0/include -isystem
> /Users/howarth/dist/x86_64-apple-darwin10.6.0/sys-include    
> -I../../gcc/libcpp -I. -I../../gcc/libcpp/../include -I../../gcc/libcpp/include
> -I/sw/include -g -O2 -mdynamic-no-pic -gtoggle -W -Wall -Wwrite-strings
> -Wmissing-format-attribute -Wstrict-prototypes -Wmissing-prototypes
> -Wold-style-definition -Wc++-compat -pedantic -Wno-long-long -Werror
> -I../../gcc/libcpp -I. -I../../gcc/libcpp/../include -I../../gcc/libcpp/include
> -I/sw/include -c -o charset.o -MT charset.o -MMD -MP -MF .deps/charset.Tpo
> ../../gcc/libcpp/charset.c
> ../../gcc/libcpp/charset.c:1:0: error: variable tracking requested, but useless
> unless producing debug info [-Werror]
> cc1: all warnings being treated as errors

I guess it is because of
  if (flag_gtoggle)
    {
      if (debug_info_level == DINFO_LEVEL_NONE)
        {
          debug_info_level = DINFO_LEVEL_NORMAL;

          if (write_symbols == NO_DEBUG)
            write_symbols = PREFERRED_DEBUGGING_TYPE;
        }
      else
        debug_info_level = DINFO_LEVEL_NONE;
    }

The conditinal in darwin.c tests debug_info_level.
I guess either this test could be moved earlier in the queue or darwin.c needs to test
gtoggle and  adjust debug_info_level accrodingly.

I am really not expert in this area. 
Joseph might know.

Honza

The regression testresults a lto-bootstrap of the proposed patch in Comment 10 on x86_64-apple-darwin10...

http://gcc.gnu.org/ml/gcc-testresults/2011-01/msg00899.html

The patch will also be problematic for powerpc-apple-darwin9 and darwin8 since it relies on dwarf2 and those OS releases still use stabs.

> The patch will also be problematic for powerpc-apple-darwin9 and darwin8 since
> it relies on dwarf2 and those OS releases still use stabs.
It should not be problem.  For stabs we don't do var tracking.
The original test tested if target debug info supports var tracking and dwarf2 is the
only one. Since stabs support is not planned, I think the new test is as good as old
one.

For that -gtoggle failure you need something like

 if (flag_var_tracking
     && (generating_for_darwin_version >= 9)
     && debug_info_level >= DINFO_LEVEL_NORMAL
     && (flag_gtoggle ? (debug_info_level >= DINFO_LEVEL_NORMAL)
	 : (debug_info_level == DINFO_LEVEL_NONE))
     && write_symbols == DWARF2_DEBUG)

(i.e. -gtoggle is supposed to replace -g by -g0 and -g0 by -g.)

Honza

I assume you really meant...

 if (flag_var_tracking
     && (generating_for_darwin_version >= 9)
     && (flag_gtoggle ? (debug_info_level >= DINFO_LEVEL_NORMAL)
         : (debug_info_level == DINFO_LEVEL_NONE))
     && write_symbols == DWARF2_DEBUG)

without the extra...

&& debug_info_level >= DINFO_LEVEL_NORMAL

Both the patch in comment 16 and 17 produce the same errors in a standard bootstrap...

../../gcc/libdecnumber/decContext.c:1:0: error: variable tracking requested, but useless unless producing debug info [-Werror]
cc1: all warnings being treated as errors

Why not retain the original code behavior unless generating lto? The patch below works with both a standard and lto-bootstrap on x86_64-apple-darwin10.

Index: gcc/config/darwin.c
===================================================================
--- gcc/config/darwin.c	(revision 168662)
+++ gcc/config/darwin.c	(working copy)
@@ -2618,7 +2618,8 @@
   if (flag_var_tracking
       && (generating_for_darwin_version >= 9)
       && debug_info_level >= DINFO_LEVEL_NORMAL
-      && debug_hooks->var_location != do_nothing_debug_hooks.var_location)
+      && (flag_generate_lto ? (write_symbols == DWARF2_DEBUG)
+      : (debug_hooks->var_location != do_nothing_debug_hooks.var_location)))
     flag_var_tracking_uninit = 1;
 
   if (MACHO_DYNAMIC_NO_PIC_P)

> Why not retain the original code behavior unless generating lto? The patch
Because the original code bahviour is wrong.  It assumes that debug_hooks is initialized
while it is not at the point.

Honza
> below works with both a standard and lto-bootstrap on x86_64-apple-darwin10.
> 
> Index: gcc/config/darwin.c
> ===================================================================
> --- gcc/config/darwin.c    (revision 168662)
> +++ gcc/config/darwin.c    (working copy)
> @@ -2618,7 +2618,8 @@
>    if (flag_var_tracking
>        && (generating_for_darwin_version >= 9)
>        && debug_info_level >= DINFO_LEVEL_NORMAL
> -      && debug_hooks->var_location != do_nothing_debug_hooks.var_location)
> +      && (flag_generate_lto ? (write_symbols == DWARF2_DEBUG)
> +      : (debug_hooks->var_location != do_nothing_debug_hooks.var_location)))
>      flag_var_tracking_uninit = 1;
> 
>    if (MACHO_DYNAMIC_NO_PIC_P)

This brute force approach works for both a standard and lto-bootstrap...

Index: gcc/config/darwin.c
===================================================================
--- gcc/config/darwin.c	(revision 168667)
+++ gcc/config/darwin.c	(working copy)
@@ -2615,10 +2615,23 @@ darwin_override_options (void)
       darwin_emit_branch_islands = true;
     }
 
+  if (flag_gtoggle)
+    {
+      if (debug_info_level == DINFO_LEVEL_NONE)
+        {
+          debug_info_level = DINFO_LEVEL_NORMAL;
+
+          if (write_symbols == NO_DEBUG)
+            write_symbols = PREFERRED_DEBUGGING_TYPE;
+        }
+      else
+        debug_info_level = DINFO_LEVEL_NONE;
+    }
+
   if (flag_var_tracking
-      && (generating_for_darwin_version >= 9)
+      && generating_for_darwin_version >= 9
       && debug_info_level >= DINFO_LEVEL_NORMAL
-      && debug_hooks->var_location != do_nothing_debug_hooks.var_location)
+      && write_symbols == DWARF2_DEBUG)
     flag_var_tracking_uninit = 1;
 
   if (MACHO_DYNAMIC_NO_PIC_P)

Created attachment 22949 [details]
proposed patch

Ok.

Created attachment 22950 [details]
revised patch

Testresults for lto-bootstrap with revised patch on x86_64-apple-darwin10...

http://gcc.gnu.org/ml/gcc-testresults/2011-01/msg01002.html

Author: rguenth
Date: Fri Jan 14 15:42:09 2011
New Revision: 168793

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=168793
Log:
2011-01-14  Jan Hubicka  <jh@suse.cz>
            Jack Howarth <howarth@bromo.med.uc.edu>

	PR target/46037
	* config/darwin.c (darwin_override_options): Honor flag_gtoggle
	when checking debug_info_level. Test write_symbols instead of
	debug_hooks->var_location when setting flag_var_tracking_uninit.

Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/config/darwin.c

Fixed at r168793.