Re: gcc compile-time performance

[dewar at gnat dot com (Robert Dewar)]

> We're supposed to be writing generic, portable code, so I don't think
> embedding such knowledge is a good idea.  What about my point about
> losing state information?

Writing generic portable code is no excuse for inefficiency. There is no
point in being over general, especially in a lexical analyzer which is
a small trivial-to-write part of a compiler where speed is important.
If new representations come up that involve new issues, you deal with
them as they come up. 

As to state information, I just don't see the problem. Have a look at the
circuits in GNAT that handle this sort of thing (Scn and its children),
they are easy to read even if you know no Ada, and tell me what's missing
there.

The interpretation of multi-byte sequences is itself quite straightforward
in terms of amount of code, though the detailed knowledge can be tricky.
I don't know the mb functions specifically, perhaps they are too 
heavy-weight for use in the lexical analyzer.

It is important to keep a lexer light and efficient, and to avoid for
example calling functions for every character, or following some high
level FSA formalism that introduces unneeded overhead.