[Bug preprocessor/53229] Fix diagnostics location when pasting tokens

[dodji at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=53229

--- Comment #2 from Dodji Seketeli <dodji at gcc dot gnu.org> 2012-05-29 09:36:34 UTC ---
Author: dodji
Date: Tue May 29 09:36:29 2012
New Revision: 187945

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=187945
Log:
PR preprocessor/53229 - Fix diagnostics location when pasting tokens

As stated in the audit trail of this problem report, consider this
test case:

    $ cat test.c
     1    struct x {
     2      int i;
     3    };
     4    struct x x;
     5
     6    #define TEST(X) x.##X
     7
     8    void foo (void)
     9    {
    10      TEST(i) = 0;
    11    }
    $

    $ cc1 -quiet test.c
    test.c: In function 'foo':
    test.c:10:1: error: pasting "." and "i" does not give a valid preprocessing
token
       TEST(i) = 0;
     ^
    $

So, when pasting tokens, the error diagnostic uses the global and
imprecise input_location variable, leading to an imprecise output.

To properly fix this, I think libcpp should keep the token of the
pasting operator '##', instead of representing it with flag on the LHS
operand's token.  That way, it could use its location.  Doing that
would be quite intrusive though.  So this patch just uses the location
of the LHS of the pasting operator, for now.  It's IMHO better than
the current situation.

The patch makes paste_tokens take a location parameter that is used in
the diagnostics.  This change can still be useful later when we can
use the location of the pasting operator, because paste_tokens will
just be passed the new, more precise location.

Incidentally, it appeared that when getting tokens from within
preprocessor directives (like what is done in gcc.dg/cpp/paste12.c),
with -ftrack-macro-expansion disabled, the location of the expansion
point of macros was being lost because
cpp_reader::set_invocation_location wasn't being properly set.  It's
because when cpp_get_token_1 calls enter_macro_context, there is a
little period of time between the beginning of that later function and
when the macro is really pushed (and thus when the macro is really
expanded) where we wrongly consider that we are not expanding the
macro because macro_of_context is still NULL.  In that period of time,
in the occurrences of indirect recursive calls to cpp_get_token_1,
this later function wrongly sets cpp_reader::invocation_location
because cpp_reader::set_invocation_location is not being properly set.

To avoid that confusion the patch does away with
cpp_reader::set_invocation_location and introduces a new flag
cpp_reader::about_to_expand_macro_p that is set in the small time
interval exposed earlier.  A new in_macro_expansion_p is introduced as
well, so that cpp_get_token_1 can now accurately detect when we are in
the process of expanding a macro, and thus correctly collect the
location of the expansion point.

People seem to like screenshots.

Thus, after the patch, we now have:

    $ cc1 -quiet test.c
    test.c: In function 'foo':
    test.c:6:18: error: pasting "." and "i" does not give a valid preprocessing
token
     #define TEST(X) x.##X
              ^
    test.c:10:3: note: in expansion of macro 'TEST'
       TEST(i) = 0;
       ^
    $

Bootstrapped and tested on x86_64-unknown-linux-gnu against trunk.

libcpp/

    PR preprocessor/53229
    * internal.h (cpp_reader::set_invocation_location): Remove.
    (cpp_reader::about_to_expand_macro_p): New member flag.
    * directives.c (do_pragma):  Remove Kludge as
    pfile->set_invocation_location is no more.
    * macro.c (cpp_get_token_1): Do away with the use of
    cpp_reader::set_invocation_location.  Just collect the macro
    expansion point when we are about to expand the top-most macro.
    Do not override cpp_reader::about_to_expand_macro_p.
    This fixes gcc.dg/cpp/paste12.c by making get_token_no_padding
    properly handle locations of expansion points.
    (cpp_get_token_with_location): Adjust, as
    cpp_reader::set_invocation_location is no more.
    (paste_tokens): Take a virtual location parameter for
    the LHS of the pasting operator.  Use it in diagnostics.  Update
    comments.
    (paste_all_tokens): Tighten the assert.  Propagate the location of
    the expansion point when no virtual locations are available.
    Pass the virtual location to paste_tokens.
    (in_macro_expansion_p): New static function.
    (enter_macro_context): Set the cpp_reader::about_to_expand_macro_p
    flag until we really start expanding the macro.

gcc/testsuite/

    PR preprocessor/53229
    * gcc.dg/cpp/paste6.c: Force to run without
    -ftrack-macro-expansion.
    * gcc.dg/cpp/paste8.c: Likewise.
    * gcc.dg/cpp/paste8-2.c: New test, like paste8.c but run with
    -ftrack-macro-expansion.
    * gcc.dg/cpp/paste12.c: Force to run without
    -ftrack-macro-expansion.
    * gcc.dg/cpp/paste12-2.c: New test, like paste12.c but run with
    -ftrack-macro-expansion.
    * gcc.dg/cpp/paste13.c: Likewise.
    * gcc.dg/cpp/paste14.c: Likewise.
    * gcc.dg/cpp/paste14-2.c: New test, like paste14.c but run with
    -ftrack-macro-expansion.
    * gcc.dg/cpp/paste18.c: New test.

Added:
    trunk/gcc/testsuite/gcc.dg/cpp/paste12-2.c
    trunk/gcc/testsuite/gcc.dg/cpp/paste14-2.c
    trunk/gcc/testsuite/gcc.dg/cpp/paste18.c
    trunk/gcc/testsuite/gcc.dg/cpp/paste8-2.c
Modified:
    trunk/gcc/testsuite/ChangeLog
    trunk/gcc/testsuite/gcc.dg/cpp/paste12.c
    trunk/gcc/testsuite/gcc.dg/cpp/paste13.c
    trunk/gcc/testsuite/gcc.dg/cpp/paste14.c
    trunk/gcc/testsuite/gcc.dg/cpp/paste6.c
    trunk/gcc/testsuite/gcc.dg/cpp/paste8.c
    trunk/libcpp/ChangeLog
    trunk/libcpp/directives.c
    trunk/libcpp/internal.h
    trunk/libcpp/macro.c