Re: [code] Factor lexer for Textadept

From: Michael Richter <>
Date: Thu, 4 Apr 2013 11:37:11 +0800

OK, I've got stack declarations much more solid now:

-- some building blocks
local maybe_some = function(p) return p^0 end
local ws =
local with_ws = function(p) return p * #ws end
local ws_with = function(p) return B(#ws) * p end
local ws_with_ws = function(p) return B( * p * #ws end

local stack_declaration = with_ws(P'(')
                        * maybe_some(1 - P'--')
                        * ws_with_ws(P'--')
                        * maybe_some(1 - P')')
                        * ws_with(P')')

This works for non-nested stack declarations exactly as expected. It, of
course, breaks down on stack declarations like "( value quote: ( word --
word ) -- word )". Attempts to build in nested stack declarations,
however, have failed because I'd have to define stack declarations in terms
of stack declarations recursively which makes the lexer get all upset and
throw up its hands while refusing to recognize *anything* at all.

Am I correct in reading to
think that I now have to bite the bullet and figure out how to use LPEG
grammar tables?

On 3 April 2013 22:43, John Benediktsson <> wrote:

> A stack declaration is ( words words words -- words words words ). It may
>>> be nested as well. So you could have ( words words ( words -- words ) --
>>> words words ). All of the words are optional, incidentally. The only
>>> fixed parts are (, ) and --. Those *must* be there.
>>> Like Forth, Factor is, indeed, whitespace delimited so ( must have
>>> whitespace before and after it as must ) and --. This is what I tried in
>>> earlier attempts to leverage, but failed with. I just don't know LPEG well
>>> enough yet to figure out even where to begin with this.
> Some other ideas:
> Can you make a stateful parser, processing tokens and when "(" is seen, go
> into "stack effect" mode, looking for ")" and consider if it is a valid
> stack effect?
> Or maybe a regexp parser, looking for something
> like "\(\s+(\S*\s+)*\-\-(\s+\S*)*\s+\)". Highlighting nested stack effects
> might require a bit more work, though.
> If it is helpful, you can see a FactorLexer[1] in Pygments that is used to
> highlight source code on Github.
> Or take a look at our VIM syntax file[2], which is fairly complete.
> Best,
> John.
> ---
> [1]
> [2]

"Perhaps people don't believe this, but throughout all of the discussions
of entering China our focus has really been what's best for the Chinese
people. It's not been about our revenue or profit or whatnot."
--Sergey Brin, demonstrating the emptiness of the "don't be evil" mantra.
You are subscribed to
To change subscription settings, send an e-mail to
To unsubscribe, send an e-mail to
Received on Wed 03 Apr 2013 - 23:37:11 EDT

This archive was generated by hypermail 2.2.0 : Thu 04 Apr 2013 - 06:36:22 EDT