protocol StatefulLexer

A lexer whose highlighting for line N depends on what came before it — the gap a per-line HighlightRule list cannot close: a block comment or a string that opens on one line and closes on another.

State is the lexer's own notion of context ("inside a /* */ comment", "inside a string opened with """, unterminated"); initialState is what an empty document — or the very first line — starts with. highlight takes the state the previous line ended in and returns both this line's spans and the state the next line should start with. Chaining that output forward, line after line, is what lets a construct span lines at all; SyntaxHighlighter's cache (see EditorView/DrawList.emitEditor) is what keeps re-chaining it on every keystroke from costing the whole file instead of just the lines a change actually invalidated.