Re: Lexer

From: anton <averbit....at.yandex.ru>
Date: Sat, 18 Jun 2011 04:14:52 -0700 (PDT)

Thanks Mitchell, but not really.

I don't see any difference between
  local preprocessor = token(l.PREPROCESSOR,#P('a') * P('ata'))
and
  local preprocessor = token(l.PREPROCESSOR, P('ata'))

I obviously looked in other lexers but I couldn't find a situation
where
one needs to lex some text in one style only if it comes before other
text,
whereas this other text should be lexed differently.

How can one for instance lex lua comment without lexing the comment
marker "--"?

I also ask about the following thing: can you please explain how this
works
local longstring = longstring * P(function(input, index)
  local level = input:match('^%[(=*)%[', index)
  if level then
    local _, stop = input:find(']'..level..']', index, true)
    return stop and stop + 1 or #input + 1
  end
end)

What type of functions does P() accept?
What are the parameters "input" and "index" and what should this
function return?

Many thanks,
anton

On 16 Jun., 04:34, mitchell <c....at.caladbolg.net> wrote:
> Hi Anton,
>
> On Wed, 15 Jun 2011, anton wrote:
> > Thanks. But sorry, I still don't get it. Why is this not working?
>
> > local preprocessor = token(l.PREPROCESSOR,#P('a') * P('ta'))
>
> This says "match the character a, but do not consume any input, then match
> the sequence 'ta'". Naturally this is impossible since text cannot match
> 'a' and 'ta' at the same position. The solution is to have
>
> > local preprocessor = token(l.PREPROCESSOR,#P('a') * P('ata'))
>
> Does this help?
>
> mitchell
>
>
>
>
>
>
>
> > _rules = {
> >  { 'whitespace', ws },
> >  { 'preprocessor', preprocessor},
> >  { 'any_char', l.any_char },
> > }
>
> > on "new sata drives"
>
> > The reference says
> > "The order of the rules is important because of the nature of LPeg.
> > LPeg tries to apply the first rule to the current position in the text
> > it is matching. If there is a match, it colors that section
> > appropriately and moves on. If there is not a match, it tries the next
> > rule, and so on."
> > What does
> > "The '#' operator matches without consuming anything"
> > mean?
> > If I have "new sata drives" string
> > LPeg will apply any_char 3 times, whitespace, any char, then
> > presumably it should apply
> > preprocessor to lex ta, and then at what position will it go on?
>
> > anton
> > P.S.: Sorry...
>
> > On 15 Jun., 15:16, Robert <ro....at.web.de> wrote:
> >> On Wed, Jun 15, 2011 at 2:58 PM, mitchell <c....at.caladbolg.net> wrote:
> >>> Hi Anton,
>
> >>> On Wed, 15 Jun 2011, anton wrote:
>
> >>>> The main problem is that if I do something wrong in a module, TA
> >>>> (well, Lua) gives a meaningful
> >>>> error message. If I do something wrong in a lexer TA simply crashes.
>
> >>> Error messages will be printed to STDOUT or STDERR in Linux.
>
> >> Also, to quote from the manual:
> >> "Poorly written lexers have the ability to crash Scintilla, so unsaved
> >> data might be lost. However, these crashes have only been observed in
> >> early lexer development, when syntax errors or pattern errors are
> >> present. Once the lexer actually starts styling text (either correctly
> >> or incorrectly; it does not matter), no crashes have occurred."
>
> >> Robert
>
> > --
> > You received this message because you are subscribed to the Google Groups "textadept" group.
> > To post to this group, send email to textadept.at.googlegroups.com.
> > To unsubscribe from this group, send email to textadept+unsubscribe.at.googlegroups.com.
> > For more options, visit this group athttp://groups.google.com/group/textadept?hl=en.
>
Received on Sat 18 Jun 2011 - 07:14:52 EDT

This archive was generated by hypermail 2.2.0 : Thu 08 Mar 2012 - 12:11:05 EST