Re: [code] lexer for indentation driven blocks

From: Mitchell <>
Date: Wed, 24 Feb 2016 08:36:56 -0500 (EST)

Hi Carl,

On Tue, 23 Feb 2016, Carl Sturtivant wrote:

> Hello Mitchell,
> On 2/22/2016 7:52 PM, Mitchell wrote:
>> It looks like you'll need a start rule that matches some level of
>> beginning whitespace and stores lexer state (that probably includes the
>> level of indentation at a minimum), and an end rule that checks for the
>> desired level of indentation before matching. This is just an idea
>> though, I'm not positive it will work in practice.
> This technique works whenever a tab containing the file it brought to
> the front, or whenever a file is opened. But not during editing. Also,
> it seems that moving between two diet template tabs causes their lexing
> to interfere with each other, as if they are sharing the saved state
> used by this technique. All of this despite my hiding that inside a
> closure for each language.

It's hard to pinpoint the cause without an example to go on. If you'd
like, I committed[1] a new `lexer.line_state` table property and
`lexer.line_from_position()` function to take the place of any `buffer`
calls. These are in the current nightly build. You can try utilizing them.
Perhaps some `buffer` references were being inadvertantly shared between

> The main problem seems to be that a lexer is only passed the whole
> document when the whole page needs lexing. I am unclear just what is
> passed to the lexer as editing continues.

The lexer is passed text starting from the beginning of the currently
edited line onward.



You are subscribed to
To change subscription settings, send an e-mail to
To unsubscribe, send an e-mail to
Received on Wed 24 Feb 2016 - 08:36:56 EST

This archive was generated by hypermail 2.2.0 : Thu 25 Feb 2016 - 06:54:26 EST