237 Commits (8b43c8419f566267a65620d1a62d7be60925fd1d)

Author SHA1 Message Date
David Majda 88c50a3e33 Add tests for zero- and one-char literals
We optimize these cases in the emitter so we should better test them.
13 years ago
David Majda 1c11e4aaa3 Split |literal| rule in the PEG.js grammar to |literal| and |string|
This is just a formality now but it will make sense later when literals
(but not strings) will allow "i" flag for case-insensitive matching.
13 years ago
David Majda cb2415a4fd Fix output printed during test failures
Expected value was being printed instead of the actual one.

Original patch by Wolfgang Kluge:

  1e875d4479
13 years ago
David Majda 17f62ae9e3 test/parser-test.js: Replace "\0" with "\x00"
Fixes the following JSHint errors:

  ./test/parser-test.js: line 353, col 54, Bad escapement.
  ./test/parser-test.js: line 384, col 54, Bad escapement.
  ./test/parser-test.js: line 436, col 60, Bad escapement.
  ./test/parser-test.js: line 437, col 60, Bad escapement.
  ./test/parser-test.js: line 472, col 50, Bad escapement.
13 years ago
David Majda 934bfa5eef test/parser-test.js: Fix missing/unnecessary semicolons
Fixes the following JSHint errors:

  ./test/parser-test.js: line 49, col 4, Missing semicolon.
  ./test/parser-test.js: line 58, col 4, Missing semicolon.
  ./test/parser-test.js: line 77, col 2, Unnecessary semicolon.
  ./test/parser-test.js: line 137, col 23, Missing semicolon.
13 years ago
David Majda d84f38c64b test/parser-test.js: Fix comments that look like nested to JSHint
Fixes the following JSHint errors:

  ./test/parser-test.js: line 521, col 29, Nested comment.
  ./test/parser-test.js: line 521, col 29, Stopping, unable to continue. (87% scanned).
13 years ago
David Majda 34d19a7dc6 test/passes-test.js: Add missing semicolons
Fixes the following JSHint errors:

  ./test/passes-test.js: line 12, col 6, Missing semicolon.
  ./test/passes-test.js: line 25, col 4, Missing semicolon.
  ./test/passes-test.js: line 229, col 41, Missing semicolon.
13 years ago
David Majda 0fe5769024 test/checks-test.js: Avoid function definitions in loops
Fixes the following JSHint errors:

  ./test/checks-test.js: line 26, col 8, Don't make functions within a loop.
  ./test/checks-test.js: line 31, col 5, Don't make functions within a loop.
  ./test/checks-test.js: line 59, col 8, Don't make functions within a loop.
  ./test/checks-test.js: line 64, col 5, Don't make functions within a loop.
13 years ago
David Majda 96192417d8 test/compiler-test.js: Remove extra comma
Fixes the following JSHint error:

  ./test/compiler-test.js: line 498, col 49, Extra comma.
13 years ago
David Majda e93eaa0e0e test/compiler-test.js: Add missing semicolon
Fixes the following JSHint error:

  ./test/compiler-test.js: line 420, col 47, Missing semicolon.
13 years ago
David Majda a92676edce Upgrade QUnit to the current master
Exact commit ID: 96e42601cadcba989f80bd4c294b7e0ee4ff1d29
13 years ago
David Majda d123cf0eda Rewrite variable handling in generated parsers
Before this commit, variables for saving match results and parse
positions in generated parsers were not used efficiently. Each rule
basically used its own variable(s) for storing the data, with names
generated sequentially during code emitting. There was no reuse of
variables and a lot of unnecessary assignments between them.

It is easy to see that both match results and parse positions can
actually be stored on a stack that grows as the parser walks deeper in
the grammar tree and shrinks as it returns. Moreover, if one creates a
new stack for each rule the parser enters, its maximum depth can be
computed statically from the grammar. This allows us to implement the
stack not as an array, but as a set of numbered variables in each
function that handles parsing of a grammar rule, avoiding potentially
slow array accesses.

This commit implements the idea from the previous paragraph, using
separate stack for match results and for parse positions. As a result,
defined variables are reused and unnecessary copying avoided.

Speed implications
------------------

This change speeds up the benchmark suite execution by 2.14%.

Detailed results (benchmark suite totals as reported by "jake benchmark"
on Node.js 0.4.8):

-----------------------------------
 Test #      Before        After
-----------------------------------
      1   129.01 kB/s   131.98 kB/s
      2   129.39 kB/s   130.13 kB/s
      3   128.63 kB/s   132.57 kB/s
      4   127.53 kB/s   129.82 kB/s
      5   127.98 kB/s   131.80 kB/s
-----------------------------------
Average   128.51 kB/s   131.26 kB/s
-----------------------------------

Size implications
-----------------

This change makes a sample of generated parsers 8.60% smaller:

Before:

  $ wc -c src/parser.js examples/*.js
   110867 src/parser.js
    13886 examples/arithmetics.js
   450125 examples/css.js
   632390 examples/javascript.js
    61365 examples/json.js
  1268633 total

After:

  $ wc -c src/parser.js examples/*.js
    99597 src/parser.js
    13077 examples/arithmetics.js
   399893 examples/css.js
   592044 examples/javascript.js
    54797 examples/json.js
  1159408 total
13 years ago
David Majda 8b2edd3c41 Remove trailing comma in parser tests (IE compatibility) 13 years ago
David Majda 747cb8afaa Reset parser position when action returns |null|
The change does not change the benchmark suite execution speed
statistically significantly.

Detailed results (benchmark suite totals as reported by "jake benchmark"
on Node.js 0.4.8):

-----------------------------------
 Test #      Before        After
-----------------------------------
      1   128.20 kB/s   128.03 kB/s
      2   130.36 kB/s   128.73 kB/s
      3   126.53 kB/s   129.72 kB/s
      4   127.46 kB/s   127.48 kB/s
      5   127.63 kB/s   128.53 kB/s
-----------------------------------
Average   128.04 kB/s   125.50 kB/s
-----------------------------------

Closes GH-25.
13 years ago
David Majda cc7ad9739f Add ability to start parsing from any grammar rule
Calling the parsing function could have been done without the ugly table
using |eval|, but this seemed to degrade performance significantly (by
about 3 %). This is probably because engines optimize badly in presence
of |eval|.

The method used in this patch does not change the benchmark suite
execution speed statistically significantly on V8.

Detailed results (benchmark suite totals):

---------------------------------
 Test #     Before       After
---------------------------------
      1   38.24 kB/s   38.28 kB/s
      2   38.35 kB/s   38.15 kB/s
      3   38.43 kB/s   38.40 kB/s
      4   38.53 kB/s   38.20 kB/s
      5   38.25 kB/s   38.39 kB/s
---------------------------------
Average   38.36 kB/s   38.39 kB/s
---------------------------------

Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.151 Safari/534.1
13 years ago
David Majda 3e7d31559d Do not pass |global| into wrapping functions in tests, it's useless 13 years ago
David Majda cfc6041041 Remove function wrapping and all |global| refrences from helpers.js
Originally I wanted to be very explicit with accesses to global object,
but since all this file is about extending it, the |global.| qualifier
seems more like noise.
13 years ago
David Majda 918dcf6ed2 Test and benchmark command-line runners can be run from any directory 13 years ago
David Majda 8f005c027b Fix encoding in |fs.readFileSync| calls ("utf-8" -> "utf8") 13 years ago
David Majda d0c074e2f8 Small style fixes 13 years ago
David Majda afcceb127f Add command-line runner for the test suite
The output format and code is heavily inspired by Nodeunit and code in
QUnit's "cli" branch.
13 years ago
David Majda 0e9d58ea96 Upgrade QUnit to the current master (c532d183664118fc2ca1) 13 years ago
David Majda aeb2cb4f1c Make sure quoting functions output only ASCII characters
This patch prevents portability problems. In particular, it fixes a
problem where "SyntaxError: Invalid range in character class." error
appeared when using command-line version on Widnows (see GH-13).
14 years ago
David Majda d493a4d143 Move test helper into its own file + reorder test file includes 14 years ago
David Majda fea6d85194 Little compiler tests clean-up 14 years ago
David Majda 8918d77da1 Add compiler passes tests (currently testing the one pass that exists) 14 years ago
David Majda 95f70c9562 Test the checks directly, do not go through |PEG.buildParser| 14 years ago
David Majda 1b75a7b9b3 Split-off |PEG.compiler.checks| tests 14 years ago
David Majda b15eb0bb5f Change term "grammar parser" to "parser" in tests 14 years ago
David Majda b105c43756 Define |global| object in different way in tests 14 years ago
David Majda 1279e87766 Simplify utility functions structure + do not export them as part of the PEG object 14 years ago
David Majda 5ab36f8018 Split the vendor directory
There are now three vendor directories. The goal is to have test- and
benchmark-specific stuff is its own directories and not in the main one.

  vendor
  test/vendor
  benchmark/vendor
14 years ago
David Majda e59f3ba338 Split the source code into several files, introduce build system
The source code is now in the src directory. The library needs to be
built using "rake", which creates the lib/peg.js file by combining the
source files.
14 years ago
David Majda 5e64d09a15 Renamed some properties of the |PEG| object
1. |PEG.Compiler| -> |PEG.compiler|
2. |PEG.grammarParser| -> |PEG.parser|

This brings us closer to the desired structure of the PEG object, which
is:

  +-PEG
    |- parser
    +- compiler
       |- checks
       |- passes
       +- emitter

These are the only things (together with the |PEG.buildParser| function
and exceptions) that I want to be publicly accessible -- as extension
points and also for easy testing of PEG.js's components.
14 years ago
David Majda 1682a25b0d Move emitter utility functions out of |PEG.Compiler| 14 years ago
David Majda d7d1a0b28c Remove unused |PEG.ArrayUtils.range| utility function 14 years ago
David Majda af1968054b Implement semantic predicates 14 years ago
David Majda 4895f4f8e4 Treat the whole grammar as an AST node 14 years ago
David Majda 917cf1cf2a Start rule of the grammar is now implicitly its first rule
Before this change, the start rule was the one named "start" and there
was an option to override that. This is now impossible.

The goal of this change is to contain all information for the parser
generation in the grammar itself.

In the future, some override directive for the start rule (like Bison's
"%start") may be added to the grammar.
14 years ago
David Majda 70cf4cd94d Reset generated variable names for each rule parsing function
Little change in the source grammar now does not change variables in all
the generated code. This is helpful especially when one has the
generated grammar stored in a VCS (this is true e.g. for our
metagrammar).
14 years ago
David Majda 66de889f4b Implement initializers 14 years ago
David Majda 718bcf5f87 Rename the |action| property of action AST nodes to |code| 14 years ago
David Majda 95735f2c97 Allow trailing semicolon (";") for rules 14 years ago
David Majda ee8c121676 Use labeled expressions and variables instead of $1, $2, etc.
Labeled expressions lead to more maintainable code and also will allow
certain optimizations (we can ignore results of expressions not passed
to the actions).

This does not speed up the benchmark suite execution statistically
significantly on V8.

Detailed results (benchmark suite totals):

---------------------------------
 Test #     Before       After
---------------------------------
      1   28.43 kB/s   28.46 kB/s
      2   28.38 kB/s   28.56 kB/s
      3   28.22 kB/s   28.58 kB/s
      4   28.76 kB/s   28.55 kB/s
      5   28.57 kB/s   28.48 kB/s
---------------------------------
Average   28.47 kB/s   28.53 kB/s
---------------------------------

Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/533.4 (KHTML, like Gecko) Chrome/5.0.375.55 Safari/533.4
14 years ago
David Majda 52704593cd Allow labeled expressions in the metagrammar (without any meaning yet) 14 years ago
David Majda 698564a3c2 Replace ":" after a rule name with "="
I'll introduce labelled expressions shortly and I want to use ":" as a
label-expression separator. This change avoids conflict between the two
meanings of ":". (What would e.g. "foo: 'bar'" mean?  Rule "foo"
matching string "bar", or string "bar" labelled "foo"?)
14 years ago
David Majda 7fdf0492c7 Fixed error message for invalid character range + added test 14 years ago
David Majda 9bf86b89a6 Fix stupid mistakes in metagrammar-test.js
The mistakes weren't caught because the first one introduces a syntax
error, causing the whole test suite not to load. Unfortunately, QUnit
didn't complain so I missed this.

The real commit these changes belong to is
33a1a7c1e9.
14 years ago
David Majda 33a1a7c1e9 Clean up class handling in the metagrammar and compiler
The class AST node now contains structured data and a raw text which is
used for error messages.
14 years ago
David Majda 137a4b4f53 Renamed |characters| -> |chars| (shorter, no loss of expressivity) 14 years ago
David Majda 76ed63c86e AST refactoring 6/6: Get rid of the |Grammar| namespace 14 years ago
David Majda b4bf49443a AST refactoring 5/6: Make AST classless 14 years ago
David Majda 1c7c5bb5da Correct variable name: "choice" -> "optional". 14 years ago
David Majda 85930cbcfe Reorder AST stuff more consistently and sensibly
There is no functional change in this commit.
14 years ago
David Majda e3aa4df090 Changed action parameter processing to avoid the arguments object.
The action now computes the number of passed parameters during the code
generation and the parameters are declared directly as $1, $2, etc. in the
generated function.

This does not speed up the benchmark suite execution statistically significantly
on V8.

Detailed results (benchmark suite totals):

---------------------------------
 Test #     Before       After
---------------------------------
      1   28.68 kB/s   29.08 kB/s
      2   28.77 kB/s   28.72 kB/s
      3   28.89 kB/s   28.78 kB/s
      4   28.84 kB/s   28.57 kB/s
      5   28.86 kB/s   28.84 kB/s
---------------------------------
Average   28.81 kB/s   28.80 kB/s
---------------------------------

Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.9 Safari/533.2
14 years ago
David Majda b2f230fad2 Added Optional AST node.
This does not speed up the benchmark suite execution statistically significantly
on V8.

Detailed results (benchmark suite totals):

---------------------------------
 Test #     Before       After
---------------------------------
      1   28.84 kB/s   28.75 kB/s
      2   28.76 kB/s   28.69 kB/s
      3   28.72 kB/s   28.69 kB/s
      4   28.84 kB/s   28.93 kB/s
      5   28.82 kB/s   28.70 kB/s
---------------------------------
Average   28.80 kB/s   28.75 kB/s
---------------------------------

Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/533.2 (KHTML, like Gecko)
Chrome/5.0.342.9 Safari/533.2
14 years ago
David Majda e5df8284b5 Added AndPredicate AST node.
This does not speed up the benchmark suite execution statistically significantly
on V8.

Detailed results (benchmark suite totals):

---------------------------------
 Test #     Before       After
---------------------------------
      1   28.72 kB/s   28.84 kB/s
      2   28.84 kB/s   28.76 kB/s
      3   28.83 kB/s   28.72 kB/s
      4   28.81 kB/s   28.84 kB/s
      5   28.76 kB/s   28.82 kB/s
---------------------------------
Average   28.79 kB/s   28.80 kB/s
---------------------------------

Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.9 Safari/533.2
14 years ago
David Majda c3c1c79665 Added OneOrMore AST node.
This speeds up the benchmark suite execution by 1.08% on V8.

Detailed results (benchmark suite totals):

---------------------------------
 Test #     Before       After
---------------------------------
      1   28.38 kB/s   28.72 kB/s
      2   28.52 kB/s   28.84 kB/s
      3   28.41 kB/s   28.83 kB/s
      4   28.47 kB/s   28.81 kB/s
      5   28.64 kB/s   28.76 kB/s
---------------------------------
Average   28.48 kB/s   28.79 kB/s
---------------------------------

Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.9 Safari/533.2
14 years ago
David Majda 3f5cb8850c Fixed PEG.buildParser's documentation and added a test. 14 years ago
David Majda 48da65d08e PEG.buildParser now accepts grammars only in string format. 14 years ago
David Majda 927f2d65c9 Exception tests also test exception messages. 14 years ago
David Majda b5ac4f0c4a Refactored helpers for testing of thrown exceptions. 14 years ago
David Majda 4b51e6a6d3 Quote null characters in regexps, IE does not like them. 14 years ago
David Majda 7fc491412d Work around the fact that IE does not recognize "\v" in strings. 14 years ago
David Majda e79e869993 Compensate for IE's lack of Array.prototype.indexOf function. 14 years ago
David Majda 05381fedfa Fixed the inverted empty character class handling test. 14 years ago
David Majda e63f64a3d5 Make the generated parsers standalone (no runtime is required).
This and also speeds up the benchmark suite execution by 7.83 % on V8.

Detailed results (benchmark suite totals):

---------------------------------
 Test #     Before       After
---------------------------------
      1   26.17 kB/s   28.16 kB/s
      2   26.05 kB/s   28.16 kB/s
      3   25.99 kB/s   28.10 kB/s
      4   26.13 kB/s   28.11 kB/s
      5   26.14 kB/s   28.07 kB/s
---------------------------------
Average   26.10 kB/s   28.14 kB/s
---------------------------------

Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.7 Safari/533.2
14 years ago
David Majda 74830d4f8f Sort expected strings in the error messages. 14 years ago
David Majda 37521cffa5 Error messages now do not contain duplicities. 14 years ago
David Majda 3291c70d97 Added \uFEFF (BOM) to the definition of whitespace in the metagrammar.
The Rhino bug that prevented inclusion of \uFEFF among the whitespace characters
is no longer relevant here because we compile character classes into regexps
now, which avoids the infinite recursion.
14 years ago
David Majda 383c5acaa6 Replaced \xA0 by \u00A0 in the whitespace definition in the metagrammar.
This is purely stylistic change.
14 years ago
David Majda f87bcd6332 Added tests for error messages displayed when a character class match fails. 14 years ago
David Majda 4f4bb34ded Implemented negative character classes (e.g. [^a-z]). 14 years ago
David Majda b3381b9352 Fixed test name. 14 years ago
David Majda 22d2ac8ac2 Rewrote implementation of classes to be regexp-based. 14 years ago
David Majda 56ffa94cc7 PEG.buildParser reports left recursion (both direct and indirect). 14 years ago
David Majda a3ecf768ca Fixed missing referenced rules test. 14 years ago
David Majda 3a65316416 PEG.buildParser reports missing referenced rules. 14 years ago
David Majda 6bbd88088b Implemented and used PEG.ArrayUtils.each. 14 years ago
David Majda 2e94dce944 Improved tests of the "arithmetics" grammar. 14 years ago
David Majda a43d1b33e3 Bootstrapped the grammar parser, yay! I should have done this long ago. 14 years ago
David Majda 636ceb2719 Metagrammar recognizes JavaScript-like comments. 14 years ago
David Majda 452243d450 Improved error reporting for predicates a bit. 14 years ago
David Majda 69906e9730 Do not recognize \uFEFF as whitespace in the metagrammar since it does not work with Rhino. 14 years ago
David Majda bddb65ab9b Improved & simplified error handling code. 14 years ago
David Majda b86a219c86 Ensure that the same grammar and start rule always generate exactly the same parser. 14 years ago
David Majda c3dd696a3e Initial commit. 14 years ago