Re: [code] how does scintilla trigger syntax highlighting on less than the whole text?

From: Cosmin Apreutesei <>
Date: Wed, 11 Sep 2013 17:51:05 +0300

Hi Mitchell,

Thanks for the explanation.

A lexer is multiple language if _LEXER._CHILDREN or _LEXER._TOKENRULES
exists, right?

> Multiple language lexers need more backtracking: they
> jump back to a whitespace token, which helps them infer which language they
> are in.

So whitespace tokens are the only tokens tagged
"<language>_whitespace" -- that's why I should go back to a whitespace
so I can extract the language name from the token name, right?

So, it seems that in scintilla, all the text after the changed bit
until eof is invalidated and must be re-parsed, with the optimization
that the lexer is stopped after the last visible line. I might need to
find a way to stop the lexer even earlier: I want to implement a
mini-map like in sublime text editor - in this case, parsing until the
last visible line of the minimap could still be too slow (too many
lines to parse). I'm thinking that I could stop the lexer once it
starts giving me the same tokens at the same positions in the
after-text. What do you think of this reasoning?

> I hope that helps,

Sure does, thanks.

You are subscribed to
To change subscription settings, send an e-mail to
To unsubscribe, send an e-mail to
Received on Wed 11 Sep 2013 - 10:51:05 EDT

This archive was generated by hypermail 2.2.0 : Thu 12 Sep 2013 - 06:43:02 EDT