[PATCH] Speed up LEX line cleaning a bit...

[Joseph S. Myers]

On Sun, 14 Mar 2010, David Miller wrote:

> From: "Joseph S. Myers" <joseph@codesourcery.com>
> Date: Sun, 14 Mar 2010 23:16:21 +0000 (UTC)
> 
> > The idea would be to reduce the number of operations done on each word 
> > while processing a line, so reducing the total number of instructions in 
> > the inner loop.  (Additionally, backslashes are common in strings, so it 
> > would be good not to have to leave the vectorized loop for them.)
> 
> Understood, but it's really not possible here since as you note we
> either need to continue supporting spaces amidst backslash escaped
> newlines, or warn about it if support is removed.

It's possible by searching backwards for a backslash or for non-whitespace 
if the vectorized loop finds \r or \n (the vectorized loop would then 
effectively return both a pointer to \r, \n or ?, and a pointer to the 
previous backslash if one found before newline).  Whether this is better 
or worse in practice than checking for backslash in the vectorized loop 
would require benchmarking to tell (just as benchmarking will be needed to 
see if the vectorization actually helps in practice on typical compiler 
input).

-- 
Joseph S. Myers
joseph@codesourcery.com