Re: Next generation of lexers

From: mitchell <mforal.n....at.gmail.com>
Date: Tue, 27 Apr 2010 07:38:25 -0700 (PDT)

Rob,

On Apr 25, 2:44 pm, Robert <ro....at.web.de> wrote:
> On Wed, Apr 21, 2010 at 6:51 PM, mitchell <mforal.n....at.gmail.com> wrote:
> >> wasn't succesful. At the moment I have this:
> >> local var_definition = 'Var' * l.space^1 * l.word * l.newline
> >> local var = '$' * l.word
> >> local variable = token('variable', var + var_definition)
>
> > local ws = token('whitespace', l.space^1)
> > local keyword = token('keyword', word_match { ... 'Var' ... })
> > local variable = #P('Var') * keyword * ws^0 * token('variable', P('$'
> > * l.word))
>
> > _rules = {
> >  ...
> >  { 'variable', variable }
> >  { 'keyword', keyword }
> >  ...
> > }
>
> > Mitchell
>
> Thank a lot for looking into this, I couldn't get it to work, however.
> Maybe I explained badly what I had in mind (I did.)
> Is it possible to have occurrences of variable names styled? Like
> local foo = 1 -- foo gets styled as a variable
> -- later
> foo = 2 -- foo gets styled as a variable
> print(foo, bar) -- foo gets styled as a variable, bar not
> Just a question, this is probably not very important to have.

In each variable token, add a functional pattern that always matches,
but grab the variable name and add it to a list. In a pattern before
identifier, check if the identifier exists in your variable list, and
if the match succeeds, match the variable token. Otherwise, don't
match and an ident will later match. Pseudocode:

local variable_list = {}
local variable = token('variable', P('Var') * P(function(patt, i)
    variable_list[patt:match('[%w_]+', i)] = true
    return i
  end) * l.word)
local potential_variable = P(function(patt, i)
    return variable_list[patt:match('[%w_]+', i)] and i or nil
  end * token('variable', l.word)
local identifier = token('identifier', l.word)

_rules = {
  ...
  { 'variable', variable },
  { 'potential_variable', potential_variable },
  { 'identifier', identifier },
  ...
}

> I put lexers for Markdown, NSIS and Lilypond up here:http://bitbucket.org/rgieseke/textadept-lexers
>
> Feel free to include the lexers, if you like, they will hopefully
> improve as I use them more. I wasn't sure if a Markdown lexer is
> against the point of Markdown, but then I liked it.

Do you think they're polished enough to be included in the upcoming
release?

> Some thoughts on the new docs:
> I had bad experiences with other editors when a lexing style changed
> theme colors so lines became impossible to read. Maybe a note could be
> added to add back- and foreground colors together? The diff lexer is
> an example for this, maybe the foreground color could be changed?
> Another issue with the diff lexer: Lua comments are treated as
> deletions. I tried some but didn't find a good solution.
>
> Another idea for the Latex lexer, maybe it would be useful to turn
> this into a Tex lexer that covers plain Tex, Latex and Context.
> Currently there are quite a lot of keywords in the Latex lexer but it
> is impossible to cover all packages or new ones defined with
> renewcommand.
> I would do a
> local command = token('keyword', '\\' * l.word)
> This would cover Context's keywords as well. Having Latex environments
> in the lexer shouldn't be a problem.
> Then I have to look some more into LuaTeX, and have the Lua lexer embedded :-)
> I like their explanation for using Lua over x:http://luatex.org/faq.html

Thanks a lot for your suggestions, but if you could have code,
patches, and/or concrete text to contribute, that would be even
better. I'm pretty strapped for time with final exams coming up and
rather than spending time trying to understand exactly what you'd
like, I'd prefer to see what exactly needs to be done. Normally I have
no problems with what you're doing :) It's just this time of year
that's difficult for me.

Best,
Mitchell

>
> Robert
>
> --
> You received this message because you are subscribed to the Google Groups "textadept" group.
> To post to this group, send email to textadept.at.googlegroups.com.
> To unsubscribe from this group, send email to textadept+unsubscribe.at.googlegroups.com.
> For more options, visit this group athttp://groups.google.com/group/textadept?hl=en.

-- 
You received this message because you are subscribed to the Google Groups "textadept" group.
To post to this group, send email to textadept.at.googlegroups.com.
To unsubscribe from this group, send email to textadept+unsubscribe.at.googlegroups.com.
For more options, visit this group at http://groups.google.com/group/textadept?hl=en.
Received on Tue 27 Apr 2010 - 10:38:25 EDT

This archive was generated by hypermail 2.2.0 : Thu 08 Mar 2012 - 11:43:39 EST