Re: [code] Fixes to Prolog and Logtalk lexers in 10.6

From: Mitchell <>
Date: Thu, 19 Sep 2019 22:11:58 -0400 (EDT)

Hi Michael,

On Fri, 20 Sep 2019, Michael Richter wrote:

> Still not working. The rules are like this:
> lex:add_rule('directive', token(lexer.WHITESPACE, lexer.starts_line(S('
> \t')^0)) *
> token(lexer.OPERATOR, P':-') *
> token(lexer.WHITESPACE, S(' \t')^0) *
> token(lexer.PREPROCESSOR,
> P(word_match(directives[dialect]))))
> -- Whitespace.
> lex:add_rule('whitespace', token(lexer.WHITESPACE,^1))
> If the directives are at the start of a line, they're all recognized,
> complete with switched dialects and all that. However put a single space
> in front of one and it switches to identifier (which is defined
> waaaaaaaaaaaaaaaaaaaaaaaaaaay later in the file!) instantly. What
> incredibly obvious thing am I misreading here?

Sorry for the trouble. I believe the whitespace rule's `^1` is the culprit. It matches both horizontal space and vertical space (newlines), so any combination of the two is matched for as long as possible. Try changing that rule to:

   lex:add_rule('whitespace', token(lexer.WHITESPACE, S(' \t')^1 + S('\r\n')^1))

It's not at all obvious :(


You are subscribed to
To change subscription settings, send an e-mail to
To unsubscribe, send an e-mail to
Received on Thu 19 Sep 2019 - 22:11:58 EDT

This archive was generated by hypermail 2.2.0 : Fri 20 Sep 2019 - 06:32:48 EDT