Bug 56231 - warning traces have bogus line information when using LTO
Summary: warning traces have bogus line information when using LTO
Status: RESOLVED FIXED
Alias: None
Product: gcc
Classification: Unclassified
Component: middle-end (show other bugs)
Version: 4.8.0
: P3 major
Target Milestone: 4.8.0
Assignee: Richard Biener
URL:
Keywords: lto
Depends on:
Blocks:
 
Reported: 2013-02-06 21:55 UTC by Matt Hargett
Modified: 2013-02-12 11:11 UTC (History)
3 users (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed: 2013-02-07 00:00:00


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Matt Hargett 2013-02-06 21:55:41 UTC
From bootstrapping GCC itself, one gets warnings that have bogus line entries, like the ":61:" line below:
In file included from ../../gcc-trunk/libbacktrace/mmapio.c:116:0,
                 from :61:
../../gcc-trunk/libbacktrace/elf.c: In function 'elf_add':
../../gcc-trunk/libbacktrace/mmapio.c:98:14: error: 'ehdr_view.len' may be used uninitialized in this function [-Werror=maybe-uninitialized]


On large LTO compilation units, the final link can include some of these kinds of warnings that contain literally hundreds of ":0:" and ":N:" entries per warning.

To reproduce the above issue, bootstrap trunk like so:
../gcc-trunk/configure --program-suffix=-4.8 --enable-languages=c,c++,lto --prefix=/u/mhargett --with-build-config=bootstrap-lto --enable-lto --with-fpmath=sse --disable-isl-version-check --disable-libmudflap --disable-libssp --enable-gold=yes --disable-multilib --disable-nls BOOT_CFLAGS="-O3 -floop-block -floop-interchange -floop-strip-mine -march=nocona =mtune=core2" CFLAGS_FOR_BUILD="-O3 -floop-block -floop-strip-mine -floop-interchange -march=nocona -mtune=core2" CXXFLAGS_FOR_BUILD="-O3 -floop-block -floop-interchange -floop-strip-mine -march=nocona -mtune=core2"

make

more complete output from the bootstrap that illustrates this bug:
/work/mhargett/gcc-trunk-obj/./prev-gcc/xg++ -B/work/mhargett/gcc-trunk-obj/./prev-gcc/ -B/u/mhargett/x86_64-unknown-linux-gnu/bin/ -nostdinc++ -B/work/mhargett/gcc-trunk-obj/prev-x86_64-unknown-linux-gnu/libstdc++-v3/src/.libs -B/work/mhargett/gcc-trunk-obj/prev-x86_64-unknown-linux-gnu/libstdc++-v3/libsupc++/.libs -I/work/mhargett/gcc-trunk-obj/prev-x86_64-unknown-linux-gnu/libstdc++-v3/include/x86_64-unknown-linux-gnu -I/work/mhargett/gcc-trunk-obj/prev-x86_64-unknown-linux-gnu/libstdc++-v3/include -I/work/mhargett/gcc-trunk/libstdc++-v3/libsupc++ -L/work/mhargett/gcc-trunk-obj/prev-x86_64-unknown-linux-gnu/libstdc++-v3/src/.libs -L/work/mhargett/gcc-trunk-obj/prev-x86_64-unknown-linux-gnu/libstdc++-v3/libsupc++/.libs   -g -O2 -O3 -march=nocona -mtune=core2 -floop-block -floop-interchange -floop-strip-mine -flto=jobserver -frandom-seed=1 -DIN_GCC   -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wmissing-format-attribute -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -Werror -fno-common  -DHAVE_CONFIG_H -static-libstdc++ -static-libgcc  gcov.o libcommon.a ../libcpp/libcpp.a   ../libbacktrace/.libs/libbacktrace.a ../libiberty/libiberty.a ../libdecnumber/libdecnumber.a  -o gcov
/work/mhargett/gcc-trunk-obj/./prev-gcc/xg++ -B/work/mhargett/gcc-trunk-obj/./prev-gcc/ -B/u/mhargett/x86_64-unknown-linux-gnu/bin/ -nostdinc++ -B/work/mhargett/gcc-trunk-obj/prev-x86_64-unknown-linux-gnu/libstdc++-v3/src/.libs -B/work/mhargett/gcc-trunk-obj/prev-x86_64-unknown-linux-gnu/libstdc++-v3/libsupc++/.libs -I/work/mhargett/gcc-trunk-obj/prev-x86_64-unknown-linux-gnu/libstdc++-v3/include/x86_64-unknown-linux-gnu -I/work/mhargett/gcc-trunk-obj/prev-x86_64-unknown-linux-gnu/libstdc++-v3/include -I/work/mhargett/gcc-trunk/libstdc++-v3/libsupc++ -L/work/mhargett/gcc-trunk-obj/prev-x86_64-unknown-linux-gnu/libstdc++-v3/src/.libs -L/work/mhargett/gcc-trunk-obj/prev-x86_64-unknown-linux-gnu/libstdc++-v3/libsupc++/.libs   -g -O2 -O3 -march=nocona -mtune=core2 -floop-block -floop-interchange -floop-strip-mine -flto=jobserver -frandom-seed=1 -DIN_GCC   -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wmissing-format-attribute -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -Werror -fno-common  -DHAVE_CONFIG_H -static-libstdc++ -static-libgcc  gcov-dump.o \
		libcommon.a ../libcpp/libcpp.a   ../libbacktrace/.libs/libbacktrace.a ../libiberty/libiberty.a ../libdecnumber/libdecnumber.a  -o gcov-dump
In file included from ../../gcc-trunk/libbacktrace/mmapio.c:116:0,
                 from :61:
../../gcc-trunk/libbacktrace/elf.c: In function 'elf_add':
../../gcc-trunk/libbacktrace/mmapio.c:98:14: error: 'ehdr_view.len' may be used uninitialized in this function [-Werror=maybe-uninitialized]
   if (munmap (const_cast.v, view->len) < 0)
              ^
In file included from ../../gcc-trunk/libbacktrace/mmapio.c:119:0,
                 from :61:
../../gcc-trunk/libbacktrace/elf.c:476:25: note: 'ehdr_view.len' was declared here
   struct backtrace_view ehdr_view;
                         ^
In file included from ../../gcc-trunk/libbacktrace/mmapio.c:116:0,
                 from :61:
../../gcc-trunk/libbacktrace/mmapio.c:98:14: error: 'ehdr_view.base' may be used uninitialized in this function [-Werror=maybe-uninitialized]
   if (munmap (const_cast.v, view->len) < 0)
              ^
In file included from ../../gcc-trunk/libbacktrace/mmapio.c:119:0,
                 from :61:
../../gcc-trunk/libbacktrace/elf.c:476:25: note: 'ehdr_view.base' was declared here
   struct backtrace_view ehdr_view;
                         ^
In file included from ../../gcc-trunk/libbacktrace/mmapio.c:119:0,
                 from :61:
../../gcc-trunk/libbacktrace/elf.c:516:10: error: 'ehdr_view.data' may be used uninitialized in this function [-Werror=maybe-uninitialized]
   memcpy (&ehdr, ehdr_view.data, sizeof ehdr);
          ^
In file included from ../../gcc-trunk/libbacktrace/mmapio.c:119:0,
                 from :61:
../../gcc-trunk/libbacktrace/elf.c:476:25: note: 'ehdr_view.data' was declared here
   struct backtrace_view ehdr_view;
                         ^
lto1: all warnings being treated as errors
make[4]: *** [/tmp/ccPsAbnW.ltrans2.ltrans.o] Error 1
lto-wrapper: make returned 2 exit status
/u/mhargett/x86_64-unknown-linux-gnu/bin/ld: lto-wrapper failed
collect2: error: ld returned 1 exit status
make[3]: *** [gcov-dump] Error 1
make[3]: *** Waiting for unfinished jobs....
In file included from ../../gcc-trunk/libiberty/cp-demangle.c:877:0,
                 from :73:
../../gcc-trunk/libbacktrace/elf.c: In function 'backtrace_initialize':
../../gcc-trunk/libbacktrace/mmapio.c:98:14: error: 'ehdr_view.len' may be used uninitialized in this function [-Werror=maybe-uninitialized]
   if (munmap (const_cast.v, view->len) < 0)
              ^
In file included from ../../gcc-trunk/libiberty/cp-demangle.c:879:0,
                 from :73:
../../gcc-trunk/libbacktrace/elf.c:476:25: note: 'ehdr_view.len' was declared here
   struct backtrace_view ehdr_view;
                         ^
In file included from ../../gcc-trunk/libiberty/cp-demangle.c:877:0,
                 from :73:
../../gcc-trunk/libbacktrace/mmapio.c:98:14: error: 'ehdr_view.base' may be used uninitialized in this function [-Werror=maybe-uninitialized]
   if (munmap (const_cast.v, view->len) < 0)
              ^
In file included from ../../gcc-trunk/libiberty/cp-demangle.c:879:0,
                 from :73:
../../gcc-trunk/libbacktrace/elf.c:476:25: note: 'ehdr_view.base' was declared here
   struct backtrace_view ehdr_view;
                         ^
In file included from ../../gcc-trunk/libiberty/cp-demangle.c:879:0,
                 from :73:
../../gcc-trunk/libbacktrace/elf.c:516:10: error: 'ehdr_view.data' may be used uninitialized in this function [-Werror=maybe-uninitialized]
   memcpy (&ehdr, ehdr_view.data, sizeof ehdr);
          ^
lto1: all warnings being treated as errors
make[4]: *** [/tmp/ccQkv0KI.ltrans13.ltrans.o] Error 1
Comment 1 Richard Biener 2013-02-07 10:52:15 UTC
Hmm, I'm not sure how the include stack works.  I suppose the reason may be
that we use

  if (file_change)
    {
      if (prev_file)
        linemap_add (line_table, LC_LEAVE, false, NULL, 0);

      linemap_add (line_table, LC_ENTER, false, data_in->current_file,
                   data_in->current_line);
    }

instead of LC_RENAME (but I'm quite positive we can't detect
"the #include directive" or "the end of the file" in any meaningful way).

Tom, can you shed some light on this?  Can we simply always use LC_RENAME?

Matt, can you try

Index: gcc/lto-streamer-in.c
===================================================================
--- gcc/lto-streamer-in.c       (revision 195841)
+++ gcc/lto-streamer-in.c       (working copy)
@@ -164,13 +164,8 @@ lto_input_location (struct bitpack_d *bp
     data_in->current_col = bp_unpack_var_len_unsigned (bp);
 
   if (file_change)
-    {
-      if (prev_file)
-       linemap_add (line_table, LC_LEAVE, false, NULL, 0);
-
-      linemap_add (line_table, LC_ENTER, false, data_in->current_file,
-                  data_in->current_line);
-    }
+    linemap_add (line_table, LC_RENAME, false, data_in->current_file,
+                data_in->current_line);
   else if (line_change)
     linemap_line_start (line_table, data_in->current_line, data_in->current_col);
 

?
Comment 2 Richard Biener 2013-02-07 13:54:20 UTC
To clarify, we are also switching between different translation-units main
filenames - but I don't think we can easily distinguish this from #includes.
Well, maybe we can, if at compile-time we can identify the "main" source
via libcpp.
Comment 3 Richard Biener 2013-02-07 14:01:50 UTC
(In reply to comment #1)
> Index: gcc/lto-streamer-in.c
> ===================================================================
> --- gcc/lto-streamer-in.c       (revision 195841)
> +++ gcc/lto-streamer-in.c       (working copy)
> @@ -164,13 +164,8 @@ lto_input_location (struct bitpack_d *bp
>      data_in->current_col = bp_unpack_var_len_unsigned (bp);
> 
>    if (file_change)
> -    {
> -      if (prev_file)
> -       linemap_add (line_table, LC_LEAVE, false, NULL, 0);
> -
> -      linemap_add (line_table, LC_ENTER, false, data_in->current_file,
> -                  data_in->current_line);
> -    }
> +    linemap_add (line_table, LC_RENAME, false, data_in->current_file,
> +                data_in->current_line);
>    else if (line_change)
>      linemap_line_start (line_table, data_in->current_line,
> data_in->current_col);

Does not work because of

const struct line_map *
linemap_add (struct line_maps *set, enum lc_reason reason,
             unsigned int sysp, const char *to_file, linenum_type to_line)
{
...
  /* When we enter the file for the first time reason cannot be
     LC_RENAME.  */
  linemap_assert (!(set->depth == 0 && reason == LC_RENAME));

trying

Index: gcc/lto-streamer-in.c
===================================================================
--- gcc/lto-streamer-in.c       (revision 195846)
+++ gcc/lto-streamer-in.c       (working copy)
@@ -165,11 +165,12 @@ lto_input_location (struct bitpack_d *bp

   if (file_change)
     {
-      if (prev_file)
-       linemap_add (line_table, LC_LEAVE, false, NULL, 0);
-
-      linemap_add (line_table, LC_ENTER, false, data_in->current_file,
-                  data_in->current_line);
+      if (!prev_file)
+       linemap_add (line_table, LC_ENTER, false, data_in->current_file,
+                    data_in->current_line);
+      else
+       linemap_add (line_table, LC_RENAME, false, data_in->current_file,
+                    data_in->current_line);
     }
   else if (line_change)
     linemap_line_start (line_table, data_in->current_line, data_in->current_col);

we'll still eventually enter a file multiple times that way.  But let's see
if it makes a difference...
Comment 4 Richard Biener 2013-02-07 14:35:24 UTC
(In reply to comment #3)
> (In reply to comment #1)
> > Index: gcc/lto-streamer-in.c
> > ===================================================================
> > --- gcc/lto-streamer-in.c       (revision 195841)
> > +++ gcc/lto-streamer-in.c       (working copy)
> > @@ -164,13 +164,8 @@ lto_input_location (struct bitpack_d *bp
> >      data_in->current_col = bp_unpack_var_len_unsigned (bp);
> > 
> >    if (file_change)
> > -    {
> > -      if (prev_file)
> > -       linemap_add (line_table, LC_LEAVE, false, NULL, 0);
> > -
> > -      linemap_add (line_table, LC_ENTER, false, data_in->current_file,
> > -                  data_in->current_line);
> > -    }
> > +    linemap_add (line_table, LC_RENAME, false, data_in->current_file,
> > +                data_in->current_line);
> >    else if (line_change)
> >      linemap_line_start (line_table, data_in->current_line,
> > data_in->current_col);
> 
> Does not work because of
> 
> const struct line_map *
> linemap_add (struct line_maps *set, enum lc_reason reason,
>              unsigned int sysp, const char *to_file, linenum_type to_line)
> {
> ...
>   /* When we enter the file for the first time reason cannot be
>      LC_RENAME.  */
>   linemap_assert (!(set->depth == 0 && reason == LC_RENAME));
> 
> trying
> 
> Index: gcc/lto-streamer-in.c
> ===================================================================
> --- gcc/lto-streamer-in.c       (revision 195846)
> +++ gcc/lto-streamer-in.c       (working copy)
> @@ -165,11 +165,12 @@ lto_input_location (struct bitpack_d *bp
> 
>    if (file_change)
>      {
> -      if (prev_file)
> -       linemap_add (line_table, LC_LEAVE, false, NULL, 0);
> -
> -      linemap_add (line_table, LC_ENTER, false, data_in->current_file,
> -                  data_in->current_line);
> +      if (!prev_file)
> +       linemap_add (line_table, LC_ENTER, false, data_in->current_file,
> +                    data_in->current_line);
> +      else
> +       linemap_add (line_table, LC_RENAME, false, data_in->current_file,
> +                    data_in->current_line);
>      }
>    else if (line_change)
>      linemap_line_start (line_table, data_in->current_line,
> data_in->current_col);
> 
> we'll still eventually enter a file multiple times that way.  But let's see
> if it makes a difference...

Doesn't make a difference.
Comment 5 Manuel López-Ibáñez 2013-02-07 19:10:17 UTC
/* Reason for creating a new line map with linemap_add.  LC_ENTER is
   when including a new file, e.g. a #include directive in C.
   LC_LEAVE is when reaching a file's end.  LC_RENAME is when a file
   name or line number changes for neither of the above reasons
   (e.g. a #line directive in C); LC_RENAME_VERBATIM is like LC_RENAME
   but a filename of "" is not specially interpreted as standard
   input. LC_ENTER_MACRO is when a macro expansion is about to start.  */

So perhaps the secret is to use just LC_ENTER?
Comment 6 rguenther@suse.de 2013-02-08 09:07:09 UTC
On Thu, 7 Feb 2013, manu at gcc dot gnu.org wrote:

> 
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56231
> 
> Manuel L?pez-Ib??ez <manu at gcc dot gnu.org> changed:
> 
>            What    |Removed                     |Added
> ----------------------------------------------------------------------------
>                  CC|                            |manu at gcc dot gnu.org
> 
> --- Comment #5 from Manuel L?pez-Ib??ez <manu at gcc dot gnu.org> 2013-02-07 19:10:17 UTC ---
> /* Reason for creating a new line map with linemap_add.  LC_ENTER is
>    when including a new file, e.g. a #include directive in C.
>    LC_LEAVE is when reaching a file's end.  LC_RENAME is when a file
>    name or line number changes for neither of the above reasons
>    (e.g. a #line directive in C); LC_RENAME_VERBATIM is like LC_RENAME
>    but a filename of "" is not specially interpreted as standard
>    input. LC_ENTER_MACRO is when a macro expansion is about to start.  */
> 
> So perhaps the secret is to use just LC_ENTER?

Hmm, I guessed that eventually the "included from" stuff is
built by nesting LC_ENTER w/o LC_LEAVE, so that probably makes it worse.

At least I see we do an initial

  linemap_add (line_table, LC_ENTER, 0, NULL, 0);

which might explain the single "included from" with bogus info.
But ISTR other frontends do sth similar.  Though now I'm curious
what breaks if we remove the above from lto/lto-lang.c
Comment 7 Richard Biener 2013-02-08 10:10:14 UTC
Testcase:

t.c:
----
volatile int b;
int foo (int *);
int main ()
{
  int a;
  b = foo (&a);
  return 0;
}

t2.c:
-----
int foo (int *a)
{
  return *a;
}

> gcc t.c t2.c -O -Wuninitialized
In file included from t.c:1:0,
                 from :2:
t.c: In function 'main':
t.c:6:5: warning: 'a' is used uninitialized in this function [-Wuninitialized]
   b = foo (&a);
     ^
In file included from t.c:1:0,
                 from :2:
t.c:5:7: note: 'a' was declared here
   int a;
       ^

with the last proposed change:

> gcc t.c t2.c -O -Wuninitialized
In file included from t.c:1:0:
t.c: In function 'main':
t.c:6:5: warning: 'a' is used uninitialized in this function [-Wuninitialized]
   b = foo (&a);
     ^
In file included from t.c:1:0:
t.c:5:7: note: 'a' was declared here
   int a;
       ^

that's still odd (the included from thing), but slightly better.

I suppose with LC_LEAVE/LC_ENTER on each file change we see we mess up
the include stack completely anyway.  So the question is if we can
suppress printing of the include stack completely from within LTO?
I don't see how we can possibly save the stack without great cost,
as we basically stream locations in random order.  The only way to
preserve it is to be able to re-construct (at compile-time where we
still have a single TU) the order in which locations were generated
and stream that info so we can replicate location construction in
exactly the same way (though we might not stream all location transitions).
But I suppose it's not worth all the trouble - the include stack is
only used for late diagnostics.

So - how do we "enter" a toplevel file correctly?  It seems that

In file included from t.c:1:0:
t.c:5:7: note: 'a' was declared here
   int a;
       ^

is because of the LC_ENTER?  Should we simply not bother about
LC_ENTER/LEAVE at all?
Comment 8 Richard Biener 2013-02-08 10:54:30 UTC
Similar (same testcase) with using

> gcc -include t.c t2.c -O2 -Wuninitialized
In file included from <command-line>:0:0:
./t.c: In function 'main':
./t.c:6:5: warning: 'a' is used uninitialized in this function [-Wuninitialized]

> gcc -include t2.c t.c -O2 -Wuninitialized
t.c: In function 'main':
t.c:6:5: warning: 'a' is used uninitialized in this function [-Wuninitialized]

but I wonder how LTO manages to get the "included from t.c" ...

tracing includes shows

 t.c
. t2.c
 t.c
 t2.c
 t.c
. t2.c
. t.c
In file included from t.c:1:0:
...

Ah, of couse - cross-LTO file we wreck tracking of current file/line/column.

I have a fix!
Comment 9 Richard Biener 2013-02-08 12:55:19 UTC
Author: rguenth
Date: Fri Feb  8 12:55:13 2013
New Revision: 195884

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=195884
Log:
2013-02-08  Richard Biener  <rguenther@suse.de>

	PR lto/56231
	* lto-streamer.h (struct data_in): Remove current_file, current_line
	and current_col members.
	* lto-streamer-out.c (lto_output_location): Stream changed bits
	en-block for efficiency.
	* lto-streamer-in.c (clear_line_info): Remove.
	(lto_input_location): Cache current file, line and column
	globally via local statics.  Read changed bits en-block.
	(input_function): Do not call clear_line_info.
	(lto_read_body): Likewise.
	(lto_input_toplevel_asms): Likewise.

	lto/
	* lto-lang.c (lto_init): Do not enter a dummy file.

Modified:
    trunk/gcc/ChangeLog
    trunk/gcc/lto-streamer-in.c
    trunk/gcc/lto-streamer-out.c
    trunk/gcc/lto-streamer.h
    trunk/gcc/lto/ChangeLog
    trunk/gcc/lto/lto-lang.c
Comment 10 Richard Biener 2013-02-08 12:55:58 UTC
Fixed.
Comment 11 Matt Hargett 2013-02-12 01:55:28 UTC
can you add the reduced test case you came up with to the testsuite? I've seen these issues come and go at various points.
Comment 12 Matt Hargett 2013-02-12 02:02:33 UTC
looking at the patch for merging elsewhere, I noticed that

 location_t
 lto_input_location (struct bitpack_d *bp, struct data_in *data_in)
 {
+  static const char *current_file;
+  static int current_line;
+  static int current_col;
   bool file_change, line_change, column_change;
   unsigned len;
-  bool prev_file = data_in->current_file != NULL;
+  bool prev_file = current_file != NULL;


current_file is potentially of unknown value on the comparison in the last line, right? or is there some static const initialization rule that implicitly initializes it to 0?
Comment 13 rguenther@suse.de 2013-02-12 11:11:11 UTC
On Tue, 12 Feb 2013, matt at use dot net wrote:

> 
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=56231
> 
> --- Comment #12 from Matt Hargett <matt at use dot net> 2013-02-12 02:02:33 UTC ---
> looking at the patch for merging elsewhere, I noticed that
> 
>  location_t
>  lto_input_location (struct bitpack_d *bp, struct data_in *data_in)
>  {
> +  static const char *current_file;
> +  static int current_line;
> +  static int current_col;
>    bool file_change, line_change, column_change;
>    unsigned len;
> -  bool prev_file = data_in->current_file != NULL;
> +  bool prev_file = current_file != NULL;
> 
> 
> current_file is potentially of unknown value on the comparison in the last
> line, right? or is there some static const initialization rule that implicitly
> initializes it to 0?

Yes, all statics are zero-initialized.

As for the testcase, the LTO testsuite harness does not support
dg-warning and friends so it's not possible to add a meaningful
testcase.