[PATCH] Speed up LEX line cleaning a bit...

[Joseph S. Myers]

On Sun, 14 Mar 2010, David Miller wrote:

> I just tossed this together and made sure it works, I haven't done any
> real substantial performance analysis.  And of course one would need
> to do this before even beginning to consider committing something like
> this :-)
> 
> In particular I am a bit concerned that 4 byte or shorter lines might
> make up a substantial portion of the lines CPP sees overall.  The
> benefit of the vectorized loop only kicks in for longer lines.

Depending on line length, it might or might not also be beneficial to work 
8 bytes at a time on 64-bit hosts.  With vector instructions it may be 
possible to check 16 bytes at a time on some hosts.  If empty lines are 
very common it may be useful to check for initial newline before trying 
the vectorized loop.

See also Zack's ideas on speeding up _cpp_clean_line that I posted in 
<http://gcc.gnu.org/ml/gcc/2007-05/msg00741.html>.  It's not clear if they 
could be effectively combined with vectorization, or what would help 
performance more, or whether several simple passes or one more complicated 
combined pass would actually be better.

You shouldn't really need to check for backslash for every character; if 
you find a newline you could then check if the line was nonempty and what 
came before was a backslash.

-- 
Joseph S. Myers
joseph@codesourcery.com