[PATCH 04/11] Fix expansion point loc for macro-like tokens

[Dodji Seketeli <dodji at redhat dot com>]

Consider the test case gcc/testsuite/gcc.dg/debug/dwarf2/pr41445-5.c.
Its interesting part is:

    #define A(x) vari x /* line 7.  */
    #define vari(x)
    #define B , varj
    int A(B) ;  /* line 10.  */

In its initial version, this test was being pre-processed as:

    # 1 "gcc/testsuite/gcc.dg/debug/dwarf2/pr41445-5.c"
    # 1 "build/gcc//"
    # 1 "<command-line>"
    # 1 "gcc/testsuite/gcc.dg/debug/dwarf2/pr41445-5.c"
    # 10 "gcc/testsuite/gcc.dg/debug/dwarf2/pr41445-5.c"
    int
    # 7 "gcc/testsuite/gcc.dg/debug/dwarf2/pr41445-5.c"
		 vari

	, varj ;

Note how "int" and "vari" are on separate lines, whereas "int" and
", varj" are on the same line.

This looks like a bug to me, even independantly from the macro
location tracking work.

With macro location tracking turned on, the preprocessed output
becomes:

    # 1 "gcc/testsuite/gcc.dg/debug/dwarf2/pr41445-5.c"
    # 1 "<command-line>"
    # 1 "gcc/testsuite/gcc.dg/debug/dwarf2/pr41445-5.c"
    # 10 "gcc/testsuite/gcc.dg/debug/dwarf2/pr41445-5.c"
    int vari , varj ;

Which, IMO, is what we'd expect.

This is due to an unexpected side effect of enter_macro_context when
passed a token that might look like a function-like macro at first
sight, but that it eventually considers to not be a macro after all.

This is the case for the "vari" token which looks like a macro when it
is first lexed, but is eventually considered to be a normal token by
enter_macro_context because it's not used as a function-like macro
invocation.

In that case, besides returning NULL, enter_macro_context sets
pfile->context->c.macro to NULL, making cpp_get_token_1 forget to set
the location of the "vari" to the expansion point of A.

Please look at the extensive comments I have added to cpp_get_token_1
to understand the details.

The easy fix here is to speculatively set the location of the returned
token before calling enter_macro_context.

Tested on x86_64-unknown-linux-gnu against trunk.  Now this test has
the same output with and without tracking locations accross macro
expansions.

Note that the bootstrap with -ftrack-macro-expansion exhibits other
separate issues that are addressed in subsequent patches.  This patch
just fixes one class of problems.

The patch does pass bootstrap with -ftrack-macro-expansion turned off,
though.

libcpp/
	* macro.c (enter_macro_context): Update comment.
	(cpp_get_token_1): Set virtual location before calling
	enter_macro_context.

gcc/testsuite/

	* gcc.dg/debug/dwarf2/pr41445-5.c: Adjust.
	* gcc.dg/debug/dwarf2/pr41445-6.c: Likewise.

---
 gcc/testsuite/gcc.dg/debug/dwarf2/pr41445-5.c |    5 ++-
 gcc/testsuite/gcc.dg/debug/dwarf2/pr41445-6.c |    5 ++-
 libcpp/macro.c                                |   43 +++++++++++++++++++++++-
 3 files changed, 49 insertions(+), 4 deletions(-)

diff --git a/gcc/testsuite/gcc.dg/debug/dwarf2/pr41445-5.c b/gcc/testsuite/gcc.dg/debug/dwarf2/pr41445-5.c
index 03af604..d21acd5 100644
--- a/gcc/testsuite/gcc.dg/debug/dwarf2/pr41445-5.c
+++ b/gcc/testsuite/gcc.dg/debug/dwarf2/pr41445-5.c
@@ -9,6 +9,9 @@
 #define B , varj
 int A(B) ;
 
-/* { dg-final { scan-assembler "DW_TAG_variable\[^\\r\\n\]*\[\\r\\n\]+\[^\\r\\n\]*\"vari\[^\\r\\n\]*DW_AT_name(\[^\\r\\n\]*\[\\r\\n\]+\[^\\r\\n\]*DW_AT_)*\[^\\r\\n\]*\[\\r\\n\]+\[^\\r\\n\]*\[^0-9a-fA-FxX](0x)?7\[^0-9a-fA-FxX]\[^\\r\\n\]*DW_AT_decl_line" } } */
+/*  We want to check that both vari and varj have the same line
+    number.  */
+
+/* { dg-final { scan-assembler "DW_TAG_variable\[^\\r\\n\]*\[\\r\\n\]+\[^\\r\\n\]*\"vari\[^\\r\\n\]*DW_AT_name(\[^\\r\\n\]*\[\\r\\n\]+\[^\\r\\n\]*DW_AT_)*\[^\\r\\n\]*\[\\r\\n\]+\[^\\r\\n\]*\[^0-9a-fA-FxX](0xa|10)\[^0-9a-fA-FxX]\[^\\r\\n\]*DW_AT_decl_line" } } */
 /* { dg-final { scan-assembler "DW_TAG_variable\[^\\r\\n\]*\[\\r\\n\]+\[^\\r\\n\]*\"varj\[^\\r\\n\]*DW_AT_name(\[^\\r\\n\]*\[\\r\\n\]+\[^\\r\\n\]*DW_AT_)*\[^\\r\\n\]*\[\\r\\n\]+\[^\\r\\n\]*\[^0-9a-fA-FxX](0xa|10)\[^0-9a-fA-FxX]\[^\\r\\n\]*DW_AT_decl_line" } } */
 /* { dg-final { cleanup-saved-temps } } */
diff --git a/gcc/testsuite/gcc.dg/debug/dwarf2/pr41445-6.c b/gcc/testsuite/gcc.dg/debug/dwarf2/pr41445-6.c
index 8aa37d1..d6d79cc 100644
--- a/gcc/testsuite/gcc.dg/debug/dwarf2/pr41445-6.c
+++ b/gcc/testsuite/gcc.dg/debug/dwarf2/pr41445-6.c
@@ -4,5 +4,8 @@
 
 #include "pr41445-5.c"
 
-/* { dg-final { scan-assembler "DW_TAG_variable\[^\\r\\n\]*\[\\r\\n\]+\[^\\r\\n\]*\"vari\[^\\r\\n\]*DW_AT_name(\[^\\r\\n\]*\[\\r\\n\]+\[^\\r\\n\]*DW_AT_)*\[^\\r\\n\]*\[\\r\\n\]+\[^\\r\\n\]*\[^0-9a-fA-FxX](0x)?7\[^0-9a-fA-FxX]\[^\\r\\n\]*DW_AT_decl_line" } } */
+/*  We want to check that both vari and varj have the same line
+    number.  */
+
+/* { dg-final { scan-assembler "DW_TAG_variable\[^\\r\\n\]*\[\\r\\n\]+\[^\\r\\n\]*\"vari\[^\\r\\n\]*DW_AT_name(\[^\\r\\n\]*\[\\r\\n\]+\[^\\r\\n\]*DW_AT_)*\[^\\r\\n\]*\[\\r\\n\]+\[^\\r\\n\]*\[^0-9a-fA-FxX](0xa|10)?\[^0-9a-fA-FxX]\[^\\r\\n\]*DW_AT_decl_line" } } */
 /* { dg-final { scan-assembler "DW_TAG_variable\[^\\r\\n\]*\[\\r\\n\]+\[^\\r\\n\]*\"varj\[^\\r\\n\]*DW_AT_name(\[^\\r\\n\]*\[\\r\\n\]+\[^\\r\\n\]*DW_AT_)*\[^\\r\\n\]*\[\\r\\n\]+\[^\\r\\n\]*\[^0-9a-fA-FxX](0xa|10)\[^0-9a-fA-FxX]\[^\\r\\n\]*DW_AT_decl_line" } } */
diff --git a/libcpp/macro.c b/libcpp/macro.c
index e0bfc31..e8ced2e 100644
--- a/libcpp/macro.c
+++ b/libcpp/macro.c
@@ -1005,8 +1005,9 @@ macro_real_token_count (const cpp_macro *macro)
    directives among macro arguments, push another context containing
    the pragma tokens before the yet-to-be-rescanned replacement list
    and return two.  Otherwise, we don't push a context and return
-   zero. LOCATION is the location of the expansion point of the
-   macro.  */
+   zero.  LOCATION is the location of the expansion point of the
+   macro.  Note that callers might want to save pfile->context as this
+   function modifies it.  */
 static int
 enter_macro_context (cpp_reader *pfile, cpp_hashnode *node,
 		     const cpp_token *result, source_location location)
@@ -2316,6 +2317,44 @@ cpp_get_token_1 (cpp_reader *pfile, source_location *location)
 	  if (pfile->state.prevent_expansion)
 	    break;
 
+	  /*  We are currently expanding the macro
+	      pfile->context->c.macro and we see that RESULT might be
+	      a macro as well.
+
+	      If we are not tracking locations accross macro
+	      expansions, then set VIRT_LOC to the expansion point of
+	      pfile->context->c.macro.
+
+	      It's useful to do this in case RESULT is finally
+	      considered to *NOT* be a macro by enter_macro_context
+	      below.  This can happen in cases like:
+
+		  #define foo(x)  // line 1
+		  foo;  // line 2
+
+	      Preprocessing the above yields "foo;" because "foo" at
+	      line 2 is not considered to be a macro; it is different
+	      from the "function-like macro foo" defined at line 1.
+
+	      So enter_macro_context below will return 0 on "foo" in
+	      this case *and* will (surprisingly!) set
+	      pfile->context->c.macro to NULL[1].  Meaning that "foo"
+	      is just a normal token, rather than a macro to be
+	      expanded.
+
+	      [1]: Note that enter_macro_context sets
+	      pfile->context->c.macro to NULL in that case because
+	      funlike_invocation_p reads one token pass "foo", sees
+	      that there is no '(' token, so we are not invoking the
+	      function-like parameter.  It then puts the tokens (which
+	      it has read after "foo") back into the tokens stream by
+	      calling _cpp_push_token_context on it, which sets
+	      pfile->context->c.macro to NULL.  */
+	  if (!CPP_OPTION (pfile, track_macro_expansion)
+	      && can_set
+	      && pfile->context->c.macro != NULL)
+	    virt_loc = pfile->invocation_location;
+
 	  /* Conditional macros require that a predicate be evaluated
 	     first.  */
 	  if ((node->flags & NODE_CONDITIONAL) != 0)
-- 
		Dodji