Very Fast: Directly Coded Lexical Analyzer

[Robert Dewar]

Ronny Peine wrote:
> Hi,
> 
> my questions is, why not use the element construction algorithm? The Thomson 
> Algorithm creates an epsilon-NFA which needs quite a lot of memory. The 
> element construction creates an NFA directly and therefor has fewer states. 
> Well, this is only interesting in the scanner creation which is not so 
> important than the scanner itself, but it can reduce the memory footprint of 
> generator. It's a pity i can't find a url for the algorithmdescription, maybe 
> i even have the wrong naming of it. I have only read it in script Compiler 
> Construction at the University.

To me, very fast (millions of lines a second) lexical analyzers are
trivial to write by hand, and I really don't see the point of tools,
and certainly not the utility of any theory in writing such code.
If anything the formalism of a finite state machine just gets in the
way, since it is more efficient to encode the state in the code
location than in data.