[15/32] new C++ lexer

[Nathan Sidwell]

Importation of header-units requires the tokenizer to recognize such 
imports during tokenization so that their macros become available.  The 
C++ parser currently (and continues to) tokenize the entire file before 
beginning c++ parsing.

This implements an explicit coroutine to manage that recognition.  It is 
used both for C++ parsing proper and for just preprocessing.  When a 
module-significant control line is observed, we call into the module 
machinery to handle it.  Usually, we'll also call again later when 
parsing that declaration.


-- 
Nathan Sidwell

-------------- next part --------------
A non-text attachment was scrubbed...
Name: 15-c++-lexer.diff
Type: text/x-patch
Size: 5633 bytes
Desc: not available
URL: <https://gcc.gnu.org/pipermail/gcc-patches/attachments/20201103/883b58de/attachment.bin>