So far, left recursion detector assumed that left recursion occurs only
when the recursive rule is at the very left-hand side of rule's
expression:
start = start
This didn't catch cases like this:
start = "a"? start
In general, if a rule reference can be reached without consuming any
input, it can lead to left recursion. This commit fixes the detector to
consider that.
Fixes#190.
Parsers can now be generated with support for tracing using the --trace
CLI option or a boolean |trace| option to |PEG.buildParser|. This makes
them trace their progress, which can be useful for debugging. Parsers
generated with tracing support are called "tracing parsers".
When a tracing parser executes, by default it traces the rules it enters
and exits by writing messages to the console. For example, a parser
built from this grammar:
start = a / b
a = "a"
b = "b"
will write this to the console when parsing input "b":
1:1 rule.enter start
1:1 rule.enter a
1:1 rule.fail a
1:1 rule.enter b
1:2 rule.match b
1:2 rule.match start
You can customize tracing by passing a custom *tracer* to parser's
|parse| method using the |tracer| option:
parser.parse(input, { trace: tracer });
This will replace the built-in default tracer (which writes to the
console) by the tracer you supplied.
The tracer must be an object with a |trace| method. This method is
called each time a tracing event happens. It takes one argument which is
an object describing the tracing event.
Currently, three events are supported:
* rule.enter -- triggered when a rule is entered
* rule.match -- triggered when a rule matches successfully
* rule.fail -- triggered when a rule fails to match
These events are triggered in nested pairs -- for each rule.enter event
there is a matching rule.match or rule.fail event.
The event object passed as an argument to |trace| contains these
properties:
* type -- event type
* rule -- name of the rule the event is related to
* offset -- parse position at the time of the event
* line -- line at the time of the event
* column -- column at the time of the event
* result -- rule's match result (only for rule.match event)
The whole tracing API is somewhat experimental (which is why it isn't
documented properly yet) and I expect it will evolve over time as
experience is gained.
The default tracer is also somewhat bare-bones. I hope that PEG.js user
community will develop more sophisticated tracers over time and I'll be
able to integrate their best ideas into the default tracer.
Rename |generateCache{Header,Footer}| to |generateRule{Header,Footer}|
and change their responsibility to generate overall header/footer of a
rule function (when optimizing for speed) or the |peg$parseRule|
function (when optimizing for speed). This creates a natural place where
to generate tracing code (coming soon).
Action and predicate code can now see variables defined in expressions
"above" them.
Based on a pull request by Bryon Vandiver (@asterick):
https://github.com/pegjs/pegjs/pull/180Fixes#316.
The |visitor.build| function now supplies default visit functions for
visitors it builds. These functions don't do anything beside traversing
the tree and passing arguments around to child visit functions.
Having the default visit functions allowed to simplify several visitors.
The TEXT instruction now replaces position at the top of the stack with
the input from that position until the current position. This is simpler
and cleaner semantics than the previous one, where TEXT also popped an
additional value from the stack and kept the position there.
Implement the following bytecode instructions:
* PUSH_UNDEFINED
* PUSH_NULL
* PUSH_FAILED
* PUSH_EMPTY_ARRAY
These instructions push simple JavaSccript values to the stack directly,
without going through constants. This makes the bytecode slightly
shorter and the bytecode generator somewhat simpler.
Also note that PUSH_EMPTY_ARRAY allows us to avoid a hack which protects
the [] constant from modification.
The |stringEscape| function both in lib/compiler/javascript.js and in
generated parsers didn't escape characters in the U+0100..U+107F and
U+1000..U+107F ranges.
Split lib/utils.js into multiple files. Some of the functions were
generic, these were moved into files in lib/utils. Other funtions were
specific for the compiler, these were moved to files in lib/compiler.
This commit only moves functions around -- there is no renaming and
cleanup performed. Both will come later.
Modules now generally store the exported object in a named variable or
function first and only assign |module.exports| at the very end. This is
a difference when compared to style used until now, where most modules
started with a |module.exports| assignment.
I think the explicit name helps readability and understandability.
Initializer code is usually indented and this indentation is carried
over to generated code. This resulted in a piece of indented code in the
middle of the parser.
This commit wraps initializer code in |{...}|, which makes indentation
in generated parsers look a bit more natural.
The action/predicate code didn't have access to the parser object. This
was mostly a side effect actions/predicates being implemented as nested
functions, in which |this| is a reference to the global object (an ugly
JavaScript quirk). The initializer, being implemented differently, had
access to the parser object via |this|, but this was not documented.
Because having access to the parser object can be useful, this commits
introduces a new |parser| variable which holds a reference to it, is
visible in action/predicate/initializer code, and is properly
documented.
See also:
https://groups.google.com/forum/#!topic/pegjs/Na7YWnz6Bmg
This is mostly done for consistency with the JavaScript example grammar,
from which the |Identifier| rule is taken from. See the previous commit
for details.
Instead of matching segments between blocks character by character,
match them as a whole. Also align the style with other similar rules
(e.g. the comment ones).
Before this commit, line continuations in character classes contributed
an empty string to the list of characters and character ranges matched
by a class. While this didn't lead to a buggy behavior with the current
code generator, the AST was wrong and the problem could have caused bugs
later.
This commit fixes the problem.
Semantic predicates are kind of |PrimaryExpression|, not kind of
|PrefixedExpression|. Therefore I extracted a rule for them and
referenced it from the |PrimaryExpression|.
Initializer and rules are now separated in a similar way as JavaScript
statements -- either by a semicolon or a line terminator, possibly with
whitespace and comments mixed in.
One consequence is that the grammars like this are now illegal:
foo = "a" bar = "b"
A semicolon needs to be inserted between the rules:
foo = "a";bar = "b"
I consider this a good change as the now-illegal syntax was somewhat
confusing.
This makes the |Primary| rule a bit more tidy. Also, matching the |.|
character really belongs to the lexical part of the grammar, next to
literals and character classes.
* Rename the |Action| rule to |CodeBlock| (it better describes what
the rule matches).
* Implement the rule in a simpler way and move it after more basic
lexical elements.
This change has two side effects:
* Label names can no longer be JavaScript reserved words.
* |$| is allowed again in label names. However, because of the
preference rules, names starting with it will be usually parsed as a
text operator followed by another identifier (denoting a rule
reference or label name).
Before this commit, whitespace was handled at the lexical level by
making tokens consume any whitespace coming after them. This was
accomplished by appending |__| to every token rule.
This commit changes whitespace handling to be more explicit. Tokens no
longer consume whitespace coming after them and syntactic rules have to
cope with it. While this slightly complicates the syntactic grammar, I
think it's a cleaner way. Moreover, it is what JavaScript example
grammar does.
One small side-effect of thich change is that the grammar is now
stand-alone (it doesn't require utils.js anymore).
When rule names are capitalized, it's easier to visually distinguish
them from non-capitalized label names. Moreover, I use capitalized rule
names in all my grammars these days.
Before this commit, a line continuation (backslash followed by a line
terminator character) contributed a character to a string or a character
class it was used in. In JavaScript and many other languages, line
continuation doesn't contribute anything.
This commit aligns PEG.js line continuation behavior with JavaScript.
Before this commit, the value of the |rawText| property of "class" AST
nodes was created in a hackish way from processed input and it didn't
always exactly represent the actual input text.
This commit changes the code so that the value of the |rawText| property
is created using the |text| function. This is a clean way which also
resolves the exact representation problem.
David Majda
David Majda