Re: [code] how does scintilla trigger syntax highlighting on less than the whole text?

From: Mitchell <>
Date: Wed, 11 Sep 2013 12:01:31 -0400 (Eastern Daylight Time)


On Wed, 11 Sep 2013, Cosmin Apreutesei wrote:

> Hi Mitchell,
> Thanks for the explanation.
> A lexer is multiple language if _LEXER._CHILDREN or _LEXER._TOKENRULES
> exists, right?

Correct, that's how you would check.

>> Multiple language lexers need more backtracking: they
>> jump back to a whitespace token, which helps them infer which language they
>> are in.
> So whitespace tokens are the only tokens tagged
> "<language>_whitespace" -- that's why I should go back to a whitespace
> so I can extract the language name from the token name, right?


> So, it seems that in scintilla, all the text after the changed bit
> until eof is invalidated and must be re-parsed, with the optimization
> that the lexer is stopped after the last visible line. I might need to
> find a way to stop the lexer even earlier: I want to implement a
> mini-map like in sublime text editor - in this case, parsing until the
> last visible line of the minimap could still be too slow (too many
> lines to parse). I'm thinking that I could stop the lexer once it
> starts giving me the same tokens at the same positions in the
> after-text. What do you think of this reasoning?

That will probably work, but figuring out if you're getting the same
tokens or not might be tricky.


You are subscribed to
To change subscription settings, send an e-mail to
To unsubscribe, send an e-mail to
Received on Wed 11 Sep 2013 - 12:01:31 EDT

This archive was generated by hypermail 2.2.0 : Thu 12 Sep 2013 - 06:43:04 EDT