On Thu, 2015-12-03 at 15:33 -0500, Jason Merrill wrote:
On 12/03/2015 09:55 AM, David Malcolm wrote:
This patch adds bulletproofing to detect purged tokens, and avoid using
them.
Alternatively, is it OK to access purged tokens for this kind of thing?
If so, would it make more sense to instead leave their locations untouched
when purging them?
I think cp_lexer_previous_token should skip past purged tokens.
Sorry if this is a silly question, but should I limit the iteration e.g.
by detecting a sentinel value? e.g.
parser->lexer->buffer->address () ?
Or is there guaranteed to be an unpurged token somewhere beforehand?
Out of interest, the prior tokens here are:
(gdb) p end_tok[0]
$25 = {type = CPP_GREATER, keyword = RID_MAX, flags = 0 '\000',
pragma_kind = PRAGMA_NONE, implicit_extern_c = 0,
error_reported = 0, purged_p = 1, location = 0, u = {tree_check_value
= 0x0, value = <tree 0x0>}}
(gdb) p end_tok[-1]
$26 = {type = CPP_NAME, keyword = RID_MAX, flags = 0 '\000', pragma_kind
= PRAGMA_NONE, implicit_extern_c = 0,
error_reported = 0, purged_p = 1, location = 0, u = {tree_check_value
= 0x0, value = <tree 0x0>}}
(gdb) p end_tok[-2]
$27 = {type = CPP_LESS, keyword = RID_MAX, flags = 0 '\000', pragma_kind
= PRAGMA_NONE, implicit_extern_c = 0,
error_reported = 0, purged_p = 1, location = 0, u = {tree_check_value
= 0x0, value = <tree 0x0>}}
(gdb) p end_tok[-3]
$28 = {type = 86, keyword = RID_MAX, flags = 1 '\001', pragma_kind =
PRAGMA_NONE, implicit_extern_c = 0, error_reported = 0,
purged_p = 0, location = 202016, u = {tree_check_value =
0x7ffff19dfd98, value = <bdver1-direct,bdver1-agu 0x7ffff19dfd98>}}
(gdb) p end_tok[-4]
$29 = {type = CPP_KEYWORD, keyword = RID_NEW, flags = 1 '\001',
pragma_kind = PRAGMA_NONE, implicit_extern_c = 0,
error_reported = 0, purged_p = 0, location = 201890, u =
{tree_check_value = 0x7ffff18a8318,
value = <identifier_node 0x7ffff18a8318 new>}}
where the previous unpurged token is:
(gdb) p end_tok[-3].purged_p
$31 = 0
(gdb) call inform (end_tok[-3].location, "")
../../src/gcc/testsuite/g++.dg/cpp0x/nsdmi-template14.C:11:14: note:
B* p = new B<N>;
^
which would give a range of:
B* p = new B<N>;
^~~~~
for the erroneous new expression, rather than:
B* p = new B<N>;
^~~~~~~~
if we used the location of the purged token (the CPP_GREATER).
I prefer the latter, hence my suggestion about not zero-ing out the
locations of tokens when purging them.