Re: [code] RFE: python and markdown lexers

From: Carlos Pita <>
Date: Tue, 3 Feb 2015 10:55:17 -0300

Thanks Mitchell,

I've written a syntax highlighter and a folding expression for
multimarkdown in vim and I know that the apparently simple grammar
could easily become a PITA. Anyway, I'll take a look and see what I
can do.

Just another lexer related question:

I tend to use org mode and rmarkdown (knitr + pandoc) to do what is
currently and fashionably called reproducible research.That is, light
markup documents with code and data interspersed with prose. In a
sense, the "negative" of literate programming.

So, would it be possible to define an lpeg grammar that combines a
number of embedded lpeg grammars (say for latex, markdown, r, python,
etc)? This also applies to the more mundane case of markdown github
style code blocks.

Take into account that I know nothing about lpeg, nor peg, nor
scintillua lexers. Even a plain yes or no answer would be helpful to
decide whether I take that trip or not.


On Mon, Feb 2, 2015 at 1:32 PM, Mitchell <> wrote:
> Hi Carlos,
> On Sun, 1 Feb 2015, Carlos Pita wrote:
>> Hi all,
>> 1) The python lexer has some shortcomings that IMO should be easy to fix:
>> - python 3 not completely supported. AFAICS, just the nonlocal keyword
>> has to be added.
> I can add this.
>> - cython not supported. This is really another lexer which should
>> inherit from the python lexer most of its functionality and add some
>> new keywords (cdef, ...). cython is an important language nowadays as
>> it's heavily used to build the python scientific stack. I think it's
>> worthwhile for scintillua to support it.
> This should be easy to do. Take a look at the "cuda" lexer, which extends
> the "cpp" lexer by adding some extra keywords and types, for inspiration.
> Feel free to submit your lexer for inclusion along with the appropriate file
> extension.
>> 2) On the markdown side:
>> - the lexer is setting font sizes, even absolute ones. I don't think
>> this should go into the lexer, am I wrong?
> You are correct, but in order to show different file sizes, a base size must
> be specified; Textadept cannot evaluate expressions in properties.
>> - support for folding at headers would make a simple and nice outliner
>> possible for note taking.
> Implementing this effectively is trickier than it looks at first glance
> since Markdown is a "2-D language" (for example, headers may be underlined).
> Take a look at the "rest" lexer, which tries to detect headers and fold
> documents properly. It's not perfect, but it may give you some ideas to
> patch the Markdown lexer. Please feel free to submit any changes you make
> for inclusion.
> Cheers,
> Mitchell
> --
> You are subscribed to
> To change subscription settings, send an e-mail to
> To unsubscribe, send an e-mail to
You are subscribed to
To change subscription settings, send an e-mail to
To unsubscribe, send an e-mail to
Received on Tue 03 Feb 2015 - 08:55:17 EST

This archive was generated by hypermail 2.2.0 : Wed 04 Feb 2015 - 06:44:28 EST