Re: [code] Fixes to Prolog and Logtalk lexers in 10.6

From: Mitchell <m.att.foicica.com>
Date: Thu, 19 Sep 2019 22:11:58 -0400 (EDT)

Hi Michael,

On Fri, 20 Sep 2019, Michael Richter wrote:

> Still not working. The rules are like this:
>
> lex:add_rule('directive', token(lexer.WHITESPACE, lexer.starts_line(S('
> \t')^0)) *
> token(lexer.OPERATOR, P':-') *
> token(lexer.WHITESPACE, S(' \t')^0) *
> token(lexer.PREPROCESSOR,
> P(word_match(directives[dialect]))))
>
> -- Whitespace.
> lex:add_rule('whitespace', token(lexer.WHITESPACE, lexer.space^1))
>
> If the directives are at the start of a line, they're all recognized,
> complete with switched dialects and all that. However put a single space
> in front of one and it switches to identifier (which is defined
> waaaaaaaaaaaaaaaaaaaaaaaaaaay later in the file!) instantly. What
> incredibly obvious thing am I misreading here?

Sorry for the trouble. I believe the whitespace rule's `lexer.space^1` is the culprit. It matches both horizontal space and vertical space (newlines), so any combination of the two is matched for as long as possible. Try changing that rule to:

   lex:add_rule('whitespace', token(lexer.WHITESPACE, S(' \t')^1 + S('\r\n')^1))

It's not at all obvious :(

Cheers,
Mitchell

-- 
You are subscribed to code.att.foicica.com.
To change subscription settings, send an e-mail to code+help.att.foicica.com.
To unsubscribe, send an e-mail to code+unsubscribe.att.foicica.com.
Received on Thu 19 Sep 2019 - 22:11:58 EDT

This archive was generated by hypermail 2.2.0 : Fri 20 Sep 2019 - 06:32:48 EDT