Re: [code] lexer for indentation driven blocks

From: Carl Sturtivant <>
Date: Wed, 24 Feb 2016 07:38:48 -0600

On 2/24/2016 7:36 AM, Mitchell wrote:
> Hi Carl,
> On Tue, 23 Feb 2016, Carl Sturtivant wrote:
>> Hello Mitchell,
>> On 2/22/2016 7:52 PM, Mitchell wrote:
>>> It looks like you'll need a start rule that matches some level of
>>> beginning whitespace and stores lexer state (that probably includes the
>>> level of indentation at a minimum), and an end rule that checks for the
>>> desired level of indentation before matching. This is just an idea
>>> though, I'm not positive it will work in practice.
>> This technique works whenever a tab containing the file it brought to
>> the front, or whenever a file is opened. But not during editing. Also,
>> it seems that moving between two diet template tabs causes their lexing
>> to interfere with each other, as if they are sharing the saved state
>> used by this technique. All of this despite my hiding that inside a
>> closure for each language.
> It's hard to pinpoint the cause without an example to go on. If you'd
> like, I committed[1] a new `lexer.line_state` table property and
> `lexer.line_from_position()` function to take the place of any `buffer`
> calls. These are in the current nightly build. You can try utilizing
> them. Perhaps some `buffer` references were being inadvertantly shared
> between buffers.
>> The main problem seems to be that a lexer is only passed the whole
>> document when the whole page needs lexing. I am unclear just what is
>> passed to the lexer as editing continues.
> The lexer is passed text starting from the beginning of the currently
> edited line onward.
> Cheers,
> Mitchell
> [1]:

Hello Mitchell, thanks for this. Probably I have enough to do the job now.


You are subscribed to
To change subscription settings, send an e-mail to
To unsubscribe, send an e-mail to
Received on Wed 24 Feb 2016 - 08:38:48 EST

This archive was generated by hypermail 2.2.0 : Thu 25 Feb 2016 - 06:54:29 EST