Re: [code] Interesting lexer challenge

From: Mitchell <>
Date: Mon, 4 Nov 2013 13:41:44 -0500 (EST)

Hi Michael,

On Mon, 4 Nov 2013, Michael Richter wrote:

> Is there any conceivable way to have the list of available "keywords" (or
> "functions" or whatever) in some way be built up dynamically as source code
> is read?
> As an example of what I mean:
> my_function(ParmA, ParmB) = foo, bar.
> When this is spotted, I'd like to be able to add "my_function" to the list
> of functions. Then later when my_function(A, B) is called, it would be
> highlighted as a function (rather than as a "fact", which it could also be).

Something like the following (untested) may be helpful:

   -- Stores custom words.
   local words = {}

   -- Matches custom words.
   local keyword = token(l.KEYWORD, P(function(input, index)
     local _, e, word = input:find('^[%w_]+', index)
     if word and words[word] then return index + e end

   -- Adds a custom word after the "function" keyword.
   local func = token(l.KEYWORD, P('function')) * ws^1 *
                token(l.FUNCTION, P(function(input, index))
     local _, e, word = input:find('^[%w_]+', index)
     if word then
       words[word] = true
       return index + e


   M._rules = {
     {'function', func},
     {'keyword', keyword},

There may be some false positives when it comes to re-lexing in the area
of a keyword definition.


You are subscribed to
To change subscription settings, send an e-mail to
To unsubscribe, send an e-mail to
Received on Mon 04 Nov 2013 - 13:41:44 EST

This archive was generated by hypermail 2.2.0 : Tue 05 Nov 2013 - 06:26:06 EST