pegjs/website/pages/documentation.html

<h1>Documentation</h1>

<h2 id="table-of-contents">Table of Contents</h2>

<ul>
  <li>
    <a href="#installation">Installation</a>
    <ul>
      <li><a href="#installation-node-js">Node.js</a></li>
      <li><a href="#installation-browser">Browser</a></li>
    </ul>
  </li>
  <li>
    <a href="#generating-a-parser">Generating a Parser</a>
    <ul>
      <li><a href="#generating-a-parser-command-line">Command Line</a></li>
      <li><a href="#generating-a-parser-javascript-api">JavaScript API</a></li>
    </ul>
  </li>
  <li><a href="#using-the-parser">Using the Parser</a></li>
  <li>
    <a href="#grammar-syntax-and-semantics">Grammar Syntax and Semantics</a>
    <ul>
      <li><a href="#grammar-syntax-and-semantics-parsing-expression-types">Parsing Expression Types</a></li>
    </ul>
  </li>
  <li><a href="#compatibility">Compatibility</a></li>
</ul>

<h2 id="installation">Installation</h2>

<h3 id="installation-node-js">Node.js</h3>

<p>To use the <code>pegjs</code> command, install PEG.js globally:</p>

<pre><code>$ npm install -g pegjs</code></pre>

<p>To use the JavaScript API, install PEG.js locally:</p>

<pre><code>$ npm install pegjs</code></pre>

<p>If you need both the <code>pegjs</code> command and the JavaScript API,
install PEG.js both ways.</p>

<h3 id="installation-browser">Browser</h3>

<p><a href="https://pegjs.org/#download">Download</a> the PEG.js library
(regular or minified version) or install it using Bower:</p>

<pre><code>$ bower install pegjs</code></pre>

<h2 id="generating-a-parser">Generating a Parser</h2>

<p>PEG.js generates parser from a grammar that describes expected input and can
specify what the parser returns (using semantic actions on matched parts of the
input). Generated parser itself is a JavaScript object with a simple API.</p>

<h3 id="generating-a-parser-command-line">Command Line</h3>

<p>To generate a parser from your grammar, use the <code>pegjs</code>
command:</p>

<pre><code>$ pegjs arithmetics.pegjs</code></pre>

<p>This writes parser source code into a file with the same name as the grammar
file but with “.js” extension. You can also specify the output file
explicitly:</p>

<pre><code>$ pegjs -o arithmetics-parser.js arithmetics.pegjs</code></pre>

<p>If you omit both input and output file, standard input and output are
used.</p>

<p>By default, the generated parser is in the Node.js module format. You can
override this using the <code>--format</code> option.</p>

<p>You can tweak the generated parser with several options:</p>

<dl>
  <dt><code>--allowed-start-rules</code></dt>
  <dd>Comma-separated list of rules the parser will be allowed to start parsing
  from (default: the first rule in the grammar).</dd>

  <dt><code>--cache</code></dt>
  <dd>Makes the parser cache results, avoiding exponential parsing time in
  pathological cases but making the parser slower.</dd>

  <dt><code>--dependency</code></dt>
  <dd>Makes the parser require a specified dependency (can be specified multiple
  times).</dd>

  <dt><code>--export-var</code></dt>
  <dd>Name of a global variable into which the parser object is assigned to when
  no module loader is detected.</dd>

  <dt><code>--extra-options</code></dt>
  <dd>Additional options (in JSON format) to pass to
  <code>peg.generate</code>.</dd>

  <dt><code>--extra-options-file</code></dt>
  <dd>File with additional options (in JSON format) to pass to
  <code>peg.generate</code>.</dd>

  <dt><code>--format</code></dt>
  <dd>Format of the generated parser: <code>amd</code>, <code>commonjs</code>,
  <code>globals</code>, <code>umd</code> (default: <code>commonjs</code>).</dd>

  <dt><code>--optimize</code></dt>
  <dd>Selects between optimizing the generated parser for parsing speed
  (<code>speed</code>) or code size (<code>size</code>) (default:
  <code>speed</code>)</dd>

  <dt><code>--plugin</code></dt>
  <dd>Makes PEG.js use a specified plugin (can be specified multiple
  times).</dd>

  <dt><code>--trace</code></dt>
  <dd>Makes the parser trace its progress.</dd>
</dl>

<h3 id="generating-a-parser-javascript-api">JavaScript API</h3>

<p>In Node.js, require the PEG.js parser generator module:</p>

<pre><code>var peg = require("pegjs");</code></pre>

<p>In browser, include the PEG.js library in your web page or application using
the <code>&lt;script&gt;</code> tag. If PEG.js detects an AMD loader, it will
define itself as a module, otherwise the API will be available in the
<code>peg</code> global object.</p>

<p>To generate a parser, call the <code>peg.generate</code> method and pass your
grammar as a parameter:</p>

<pre><code>var parser = peg.generate("start = ('a' / 'b')+");</code></pre>

<p>The method will return generated parser object or its source code as a string
(depending on the value of the <code>output</code> option — see below). It will
throw an exception if the grammar is invalid. The exception will contain
<code>message</code> property with more details about the error.</p>

<p>You can tweak the generated parser by passing a second parameter with an
options object to <code>peg.generate</code>. The following options are
supported:</p>

<dl>
  <dt><code>allowedStartRules</code></dt>
  <dd>Rules the parser will be allowed to start parsing from (default: the first
  rule in the grammar).</dd>

  <dt><code>cache</code></dt>
  <dd>If <code>true</code>, makes the parser cache results, avoiding exponential
  parsing time in pathological cases but making the parser slower (default:
  <code>false</code>).</dd>

  <dt><code>dependencies</code></dt>
  <dd>Parser dependencies, the value is an object which maps variables used to
  access the dependencies in the parser to module IDs used to load them; valid
  only when <code>format</code> is set to <code>"amd"</code>,
  <code>"commonjs"</code>, or <code>"umd"</code> (default:
  <code>{}</code>).</dd>

  <dt><code>exportVar</code></dt>
  <dd>Name of a global variable into which the parser object is assigned to when
  no module loader is detected; valid only when <code>format</code> is set to
  <code>"globals"</code> or <code>"umd"</code> (default:
  <code>null</code>).</dd>

  <dt><code>format</code></dt>
  <dd>format of the generated parser (<code>"amd"</code>, <code>"bare"</code>,
  <code>"commonjs"</code>, <code>"globals"</code>, or <code>"umd"</code>); valid
  only when <code>output</code> is set to <code>"source"</code> (default:
  <code>"bare"</code>).</dd>

  <dt><code>optimize</code></dt>
  <dd>Selects between optimizing the generated parser for parsing speed
  (<code>"speed"</code>) or code size (<code>"size"</code>) (default:
  <code>"speed"</code>).</dd>

  <dt><code>output</code></dt>
  <dd>If set to <code>"parser"</code>, the method will return generated parser
  object; if set to <opde>"source"</code>, it will return parser source code as
  a string (default: <code>"parser"</code>).</dd>

  <dt><code>plugins</code></dt>
  <dd>Plugins to use.</dd>

  <dt><code>trace</code></dt>
  <dd>Makes the parser trace its progress (default: <code>false</code>).</dd>
</dl>

<h2 id="using-the-parser">Using the Parser</h2>

<p>Using the generated parser is simple &mdash; just call its <code>parse</code>
method and pass an input string as a parameter. The method will return a parse
result (the exact value depends on the grammar used to generate the parser) or
throw an exception if the input is invalid. The exception will contain
<code>location</code>, <code>expected</code>, <code>found</code> and
<code>message</code> properties with more details about the error.</p>

<pre><code>parser.parse("abba"); // returns ["a", "b", "b", "a"]

parser.parse("abcd"); // throws an exception </code></pre>

<p>You can tweak parser behavior by passing a second parameter with an options
object to the <code>parse</code> method. The following options are
supported:</p>

<dl>
  <dt><code>startRule</code></dt>
  <dd>Name of the rule to start parsing from.</dd>

  <dt><code>tracer</code></dt>
  <dd>Tracer to use.</dd>
</dl>

<p>Parsers can also support their own custom options.</p>

<h2 id="grammar-syntax-and-semantics">Grammar Syntax and Semantics</h2>

<p>The grammar syntax is similar to JavaScript in that it is not line-oriented
and ignores whitespace between tokens. You can also use JavaScript-style
comments (<code>// ...</code> and <code>/* ... */</code>).</p>

<p>Let's look at example grammar that recognizes simple arithmetic expressions
like <code>2*(3+4)</code>. A parser generated from this grammar computes their
values.</p>

<pre><code>start
  = additive

additive
  = left:multiplicative "+" right:additive { return left + right; }
  / multiplicative

multiplicative
  = left:primary "*" right:multiplicative { return left * right; }
  / primary

primary
  = integer
  / "(" additive:additive ")" { return additive; }

integer "integer"
  = digits:[0-9]+ { return parseInt(digits.join(""), 10); }</code></pre>

<p>On the top level, the grammar consists of <em>rules</em> (in our example,
there are five of them). Each rule has a <em>name</em> (e.g.
<code>integer</code>) that identifies the rule, and a <em>parsing
expression</em> (e.g. <code>digits:[0-9]+ { return parseInt(digits.join(""),
10); }</code>) that defines a pattern to match against the input text and
possibly contains some JavaScript code that determines what happens when the
pattern matches successfully. A rule can also contain <em>human-readable
name</em> that is used in error messages (in our example, only the
<code>integer</code> rule has a human-readable name). The parsing starts at the
first rule, which is also called the <em>start rule</em>.</p>

<p>A rule name must be a JavaScript identifier. It is followed by an equality
sign (“=”) and a parsing expression. If the rule has a human-readable name, it
is written as a JavaScript string between the name and separating equality sign.
Rules need to be separated only by whitespace (their beginning is easily
recognizable), but a semicolon (“;”) after the parsing expression is
allowed.</p>

<p>The first rule can be preceded by an <em>initializer</em> &mdash; a piece of
JavaScript code in curly braces (“{” and “}”). This code is executed before the
generated parser starts parsing. All variables and functions defined in the
initializer are accessible in rule actions and semantic predicates. The code
inside the initializer can access options passed to the parser using the
<code>options</code> variable. Curly braces in the initializer code must be
balanced. Let's look at the example grammar from above using a simple
initializer.</p>

<pre><code>{
  function makeInteger(o) {
    return parseInt(o.join(""), 10);
  }
}

start
  = additive

additive
  = left:multiplicative "+" right:additive { return left + right; }
  / multiplicative

multiplicative
  = left:primary "*" right:multiplicative { return left * right; }
  / primary

primary
  = integer
  / "(" additive:additive ")" { return additive; }

integer "integer"
  = digits:[0-9]+ { return makeInteger(digits); }</code></pre>

<p>The parsing expressions of the rules are used to match the input text to the
grammar. There are various types of expressions &mdash; matching characters or
character classes, indicating optional parts and repetition, etc. Expressions
can also contain references to other rules. See <a
href="#grammar-syntax-and-semantics-parsing-expression-types">detailed
description below</a>.</p>

<p>If an expression successfully matches a part of the text when running the
generated parser, it produces a <em>match result</em>, which is a JavaScript
value. For example:</p>

<ul>
  <li>An expression matching a literal string produces a JavaScript string
  containing matched text.</li>

  <li>An expression matching repeated occurrence of some subexpression produces
  a JavaScript array with all the matches.</li>
</ul>

<p>The match results propagate through the rules when the rule names are used in
expressions, up to the start rule. The generated parser returns start rule's
match result when parsing is successful.</p>

<p>One special case of parser expression is a <em>parser action</em> &mdash; a
piece of JavaScript code inside curly braces (“{” and “}”) that takes match
results of some of the the preceding expressions and returns a JavaScript value.
This value is considered match result of the preceding expression (in other
words, the parser action is a match result transformer).</p>

<p>In our arithmetics example, there are many parser actions. Consider the
action in expression <code>digits:[0-9]+ { return parseInt(digits.join(""), 10);
}</code>. It takes the match result of the expression [0-9]+, which is an array
of strings containing digits, as its parameter. It joins the digits together to
form a number and converts it to a JavaScript <code>number</code> object.</p>

<h3 id="grammar-syntax-and-semantics-parsing-expression-types">Parsing Expression Types</h3>

<p>There are several types of parsing expressions, some of them containing
subexpressions and thus forming a recursive structure:</p>

<dl>
  <dt><code>"<em>literal</em>"<br>'<em>literal</em>'</code></dt>

  <dd>
    <p>Match exact literal string and return it. The string syntax is the same
    as in JavaScript. Appending <code>i</code> right after the literal makes the
    match case-insensitive.</p>
  </dd>

  <dt><code>.</code></dt>

  <dd>
    <p>Match exactly one character and return it as a string.</p>
  </dd>

  <dt><code>[<em>characters</em>]</code></dt>

  <dd>
    <p>Match one character from a set and return it as a string. The characters
    in the list can be escaped in exactly the same way as in JavaScript string.
    The list of characters can also contain ranges (e.g. <code>[a-z]</code>
    means “all lowercase letters”). Preceding the characters with <code>^</code>
    inverts the matched set (e.g. <code>[^a-z]</code> means “all character but
    lowercase letters”). Appending <code>i</code> right after the literal makes
    the match case-insensitive.</p>
  </dd>

  <dt><code><em>rule</em></code></dt>

  <dd>
    <p>Match a parsing expression of a rule recursively and return its match
    result.</p>
  </dd>

  <dt><code>( <em>expression</em> )</code></dt>

  <dd>
    <p>Match a subexpression and return its match result.</p>
  </dd>

  <dt><code><em>expression</em> *</code></dt>

  <dd>
    <p>Match zero or more repetitions of the expression and return their match
    results in an array. The matching is greedy, i.e. the parser tries to match
    the expression as many times as possible. Unlike in regular expressions,
    there is no backtracking.</p>
  </dd>

  <dt><code><em>expression</em> +</code></dt>

  <dd>
    <p>Match one or more repetitions of the expression and return their match
    results in an array. The matching is greedy, i.e. the parser tries to match
    the expression as many times as possible. Unlike in regular expressions,
    there is no backtracking.</p>
  </dd>

  <dt><code><em>expression</em> ?</code></dt>

  <dd>
    <p>Try to match the expression. If the match succeeds, return its match
    result, otherwise return <code>null</code>. Unlike in regular expressions,
    there is no backtracking.</p>
  </dd>

  <dt><code>&amp; <em>expression</em></code></dt>

  <dd>
    <p>Try to match the expression. If the match succeeds, just return
    <code>undefined</code> and do not consume any input, otherwise consider the
    match failed.</p>
  </dd>

  <dt><code>! <em>expression</em></code></dt>

  <dd>
    <p>Try to match the expression. If the match does not succeed, just return
    <code>undefined</code> and do not consume any input, otherwise consider the
    match failed.</p>
  </dd>

  <dt><code>&amp; { <em>predicate</em> }</code></dt>

  <dd>
    <p>The predicate is a piece of JavaScript code that is executed as if it was
    inside a function. It gets the match results of labeled expressions in
    preceding expression as its arguments. It should return some JavaScript
    value using the <code>return</code> statement. If the returned value
    evaluates to <code>true</code> in boolean context, just return
    <code>undefined</code> and do not consume any input; otherwise consider the
    match failed.</p>

    <p>The code inside the predicate can access all variables and functions
    defined in the initializer at the beginning of the grammar.</p>

    <p>The code inside the predicate can also access location information using
    the <code>location</code> function. It returns an object like this:</p>

    <pre><code>{
  start: { offset: 23, line: 5, column: 6 },
  end:   { offset: 23, line: 5, column: 6 }
}</code></pre>

    <p>The <code>start</code> and <code>end</code> properties both refer to the
    current parse position. The <code>offset</code> property contains an offset
    as a zero-based index and <code>line</code> and <code>column</code>
    properties contain a line and a column as one-based indices.</p>

    <p>The code inside the predicate can also access options passed to the
    parser using the <code>options</code> variable.</p>

    <p>Note that curly braces in the predicate code must be balanced.</p>
  </dd>

  <dt><code>! { <em>predicate</em> }</code></dt>

  <dd>
    <p>The predicate is a piece of JavaScript code that is executed as if it was
    inside a function. It gets the match results of labeled expressions in
    preceding expression as its arguments. It should return some JavaScript
    value using the <code>return</code> statement. If the returned value
    evaluates to <code>false</code> in boolean context, just return
    <code>undefined</code> and do not consume any input; otherwise consider the
    match failed.</p>

    <p>The code inside the predicate can access all variables and functions
    defined in the initializer at the beginning of the grammar.</p>

    <p>The code inside the predicate can also access location information using
    the <code>location</code> function. It returns an object like this:</p>

    <pre><code>{
  start: { offset: 23, line: 5, column: 6 },
  end:   { offset: 23, line: 5, column: 6 }
}</code></pre>

    <p>The <code>start</code> and <code>end</code> properties both refer to the
    current parse position. The <code>offset</code> property contains an offset
    as a zero-based index and <code>line</code> and <code>column</code>
    properties contain a line and a column as one-based indices.</p>

    <p>The code inside the predicate can also access options passed to the
    parser using the <code>options</code> variable.</p>

    <p>Note that curly braces in the predicate code must be balanced.</p>
  </dd>

  <dt><code>$ <em>expression</em></code></dt>

  <dd>
    <p>Try to match the expression. If the match succeeds, return the matched
    text instead of the match result.</p>
  </dd>

  <dt><code><em>label</em> : <em>expression</em></code></dt>

  <dd>
    <p>Match the expression and remember its match result under given label.
    The label must be a JavaScript identifier.</p>

    <p>Labeled expressions are useful together with actions, where saved match
    results can be accessed by action's JavaScript code.</p>
  </dd>

  <dt><code><em>expression<sub>1</sub></em> <em>expression<sub>2</sub></em> ...  <em>expression<sub>n</sub></em></code></dt>

  <dd>
    <p>Match a sequence of expressions and return their match results in an array.</p>
  </dd>

  <dt><code><em>expression</em> { <em>action</em> }</code></dt>

  <dd>
    <p>Match the expression. If the match is successful, run the action,
    otherwise consider the match failed.</p>

    <p>The action is a piece of JavaScript code that is executed as if it was
    inside a function. It gets the match results of labeled expressions in
    preceding expression as its arguments. The action should return some
    JavaScript value using the <code>return</code> statement. This value is
    considered match result of the preceding expression.</p>

    <p>To indicate an error, the code inside the action can invoke the
    <code>expected</code> function, which makes the parser throw an exception.
    The function takes two parameters — a description of what was expected at
    the current position and optional location information (the default is what
    <code>location</code> would return — see below). The description will be
    used as part of a message of the thrown exception.</p>

    <p>The code inside an action can also invoke the <code>error</code>
    function, which also makes the parser throw an exception. The function takes
    two parameters — an error message and optional location information (the
    default is what <code>location</code> would return — see below). The message
    will be used by the thrown exception.</p>

    <p>The code inside the action can access all variables and functions defined
    in the initializer at the beginning of the grammar. Curly braces in the
    action code must be balanced.</p>

    <p>The code inside the action can also access the text matched by the
    expression using the <code>text</code> function.</p>

    <p>The code inside the action can also access location information using the
    <code>location</code> function. It returns an object like this:</p>

    <pre><code>{
  start: { offset: 23, line: 5, column: 6 },
  end:   { offset: 25, line: 5, column: 8 }
}</code></pre>

    <p>The <code>start</code> property refers to the position at the beginning
    of the expression, the <code>end</code> property refers to position after
    the end of the expression. The <code>offset</code> property contains an
    offset as a zero-based index and <code>line</code> and <code>column</code>
    properties contain a line and a column as one-based indices.</p>

    <p>The code inside the action can also access options passed to the parser
    using the <code>options</code> variable.</p>

    <p>Note that curly braces in the action code must be balanced.</p>
  </dd>

  <dt><code><em>expression<sub>1</sub></em> / <em>expression<sub>2</sub></em> / ... / <em>expression<sub>n</sub></em></code></dt>

  <dd>
    <p>Try to match the first expression, if it does not succeed, try the second
    one, etc. Return the match result of the first successfully matched
    expression. If no expression matches, consider the match failed.</p>
  </dd>
</dl>

<h2 id="compatibility">Compatibility</h2>

<p>Both the parser generator and generated parsers should run well in the
following environments:</p>

<ul>
  <li>Node.js 0.10.0+</li>
  <li>Internet Explorer 8+</li>
  <li>Edge</li>
  <li>Firefox</li>
  <li>Chrome</li>
  <li>Safari</li>
  <li>Opera</li>
</ul>