Re: [code] Fixes to Prolog and Logtalk lexers in 10.6

From: Mitchell <m.att.foicica.com>
Date: Wed, 18 Sep 2019 09:25:38 -0400 (EDT)

Hi Michael,

On Wed, 18 Sep 2019, Michael Richter wrote:

> I'm in the process of making a more sophisticated lexer for Prolog/Logtalk,
> Mitchell, and came across a problem I can't quite fix.
>
> In Prolog and its workalikes, a "directive" looks like this:
>
> :- directive.
> :- another_directive_with_arguments(foo, bar).
>
> I want to only syntax-colour the words "directive" and
> "another_directive_with_arguments". So the rule is, basically, the token
> ":-", a variable number of spaces (including 0), and then one of a list of
> words that can be directives (using word_match[[...]] in this case).
>
> I can't get this to work. The pattern that seems like it should work
> doesn't:
>
> lex:add_rule('directive', token(lexer.PREPROCESSOR, B(P':-') *
> lexer.space^0 * word_match[[...]]))
>
> There's no complaint from the parser, but the following won't be recognized
> as a directive:
>
> :- use_module(foo).
>
> This instead has use_module recognized as an "identifier" (defined
> waaaaaaaaaaaaaaaaaaaaaaaaaaaay after this rule) and foo identified as a
> keyword (dummy keyword thrown in for testing another rule).
>
> I'm at a loss. How do I get the behaviour I'm expecting?

You could try adding this before the 'operator' rule:

   lex:add_rule('directive', token(lexer.PREPROCESSOR, ':-' * lexer.space^0 * word_match[[foo bar baz]]))

Note that I've found `lpeg.B()` to not be very reliable in lexers. It's better to just start matching what you want than to match a piece and then look backwards.

Cheers,
Mitchell

>
> On Wed, 18 Sep 2019 at 01:00, Mitchell <m.att.foicica.com> wrote:
>
>> Hi Michael,
>>
>> On Tue, 17 Sep 2019, Michael Richter wrote:
>>
>>> In the process of updating Logtalk support, I found a few ugly oversights
>>> in the Prolog support. The lexers in this ZIP file are the first pass
>> at a
>>> fix. All "keywords", operators, and numerical forms are now properly
>>> supported (validated against a test file contained in the Logtalk
>>> distribution). In the longer term, because of some of the issues
>> involved
>>> in just what a "keyword" really is, the lexers will both be replaced with
>>> ones that handle things according to Prolog's semantics (which are really
>>> weird to people unused to how the Prolog world works).
>>>
>>> For now, however, this will cover more than enough to be usable. I don't
>>> have time at the moment for the major research and surgery process that
>>> will be required to properly cover "keywords".
>>
>> Thanks, I'll apply your changes when I get the chance.
>>
>> Cheers,
>> Mitchell
>> --
>> You are subscribed to code.att.foicica.com.
>> To change subscription settings, send an e-mail to code+help.att.foicica.com.
>> To unsubscribe, send an e-mail to code+unsubscribe.att.foicica.com.
>>
>>
>
>

Mitchell

-- 
You are subscribed to code.att.foicica.com.
To change subscription settings, send an e-mail to code+help.att.foicica.com.
To unsubscribe, send an e-mail to code+unsubscribe.att.foicica.com.
Received on Wed 18 Sep 2019 - 09:25:38 EDT

This archive was generated by hypermail 2.2.0 : Thu 19 Sep 2019 - 06:47:08 EDT