[code] Markdown lexer

From: Alexander Misel <alexander_misel.att.live.cn>
Date: Thu, 28 Nov 2019 11:03:48 +0000

Hi Mitchell,

I studied the markdown lexer again and made a few changes again. Link of my version:
https://github.com/AlexanderMisel/ta-tweaks/blob/master/markdown.lua

In markdown grammar, we often need to check several inline spaces followed by a newline or EOF, so I added
local inline_space = S('\t\v\f\r ')
local blank_line = inline_space^0 * (P('\n') + -1)
and used in
token('code', lexer.starts_line(P('```')) * (lexer.any - '```')^0 * P('```' * blank_line)^-1))
to color the background till the line end of the last line of code block, and in
lex:add_rule('inline_code',
             token('code', P('``') * (lexer.any - ('\n' * blank_line) - '``')^0 * P('``')^-1 +
                           P('`') * (lexer.any - ('\n' * blank_line) - '`')^0 * P('`')^-1))
to assure that it cannot contain an empty line. We cannot have an blank line in flanked range either, so the return becomes
left_fl * (lexer.any - ('\n' * blank_line)
                     - (not_inword and s * #punct_space or s))^0 * right_fl

I also found that loading markdown lexer at the first time would be slower than other languages. And the cause seems to be these rules
lex:add_rule('header',
             token('h6', lexer.starts_line('######') * lexer.nonnewline^0) +
...
lex:add_style('h1', hstyle..',size:'..(font_size + 5))

-- 
You are subscribed to code.att.foicica.com.
To change subscription settings, send an e-mail to code+help.att.foicica.com.
To unsubscribe, send an e-mail to code+unsubscribe.att.foicica.com.
Received on Thu 28 Nov 2019 - 06:03:48 EST

This archive was generated by hypermail 2.2.0 : Thu 28 Nov 2019 - 06:41:23 EST