joepie91
/
pegjs
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
redux
master
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '6d25da2d2b'
${ noResults }
pegjs/lib/compiler/passes/generate-js.js

1827 lines
61 KiB
JavaScript
Raw Normal View History Unescape Escape

Optional features This commit enables optional features that are enabled by default in the generated parser. For now, only some of the helpers and filename are generated based on this new option, but this will change in the future most likely. Resolves #421
7 years ago
/* eslint no-mixed-operators: 0, prefer-const: 0, eqeqeq: 0 */
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
Convert PEG.js code to strict mode * Issues #323
10 years ago
"use strict";
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
const js = require( "../js" );
Code generator rewrite This is a complete rewrite of the PEG.js code generator. Its goals are: 1. Allow optimizing the generated parser code for code size as well as for parsing speed. 2. Prepare ground for future optimizations and big features (like incremental parsing). 2. Replace the old template-based code-generation system with something more lightweight and flexible. 4. General code cleanup (structure, style, variable names, ...). New Architecture ---------------- The new code generator consists of two steps: * Bytecode generator -- produces bytecode for an abstract virtual machine * JavaScript generator -- produces JavaScript code based on the bytecode The abstract virtual machine is stack-based. Originally I wanted to make it register-based, but it turned out that all the code related to it would be more complex and the bytecode itself would be longer (because of explicit register specifications in instructions). The only downsides of the stack-based approach seem to be few small inefficiencies (see e.g. the |NIP| instruction), which seem to be insignificant. The new generator allows optimizing for parsing speed or code size (you can choose using the |optimize| option of the |PEG.buildParser| method or the --optimize/-o option on the command-line). When optimizing for size, the JavaScript generator emits the bytecode together with its constant table and a generic bytecode interpreter. Because the interpreter is small and the bytecode and constant table grow only slowly with size of the grammar, the resulting parser is also small. When optimizing for speed, the JavaScript generator just compiles the bytecode into JavaScript. The generated code is relatively efficient, so the resulting parser is fast. Internal Identifiers -------------------- As a small bonus, all internal identifiers visible to user code in the initializer, actions and predicates are prefixed by |peg$|. This lowers the chance that identifiers in user code will conflict with the ones from PEG.js. It also makes using any internals in user code ugly, which is a good thing. This solves GH-92. Performance ----------- The new code generator improved parsing speed and parser code size significantly. The generated parsers are now: * 39% faster when optimizing for speed * 69% smaller when optimizing for size (without minification) * 31% smaller when optimizing for size (with minification) (Parsing speed was measured using the |benchmark/run| script. Code size was measured by generating parsers for examples in the |examples| directory and adding up the file sizes. Minification was done by |uglify --ascii| in version 1.3.4.) Final Note ---------- This is just a beginning! The new code generator lays a foundation upon which many optimizations and improvements can (and will) be made. Stay tuned :-)
12 years ago
Use only "//" comments See #443.
8 years ago
// Generates parser JavaScript code.
Added the Session API
7 years ago
function generateJS( ast, session, options ) {
const op = session.opcodes;
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
Optional features This commit enables optional features that are enabled by default in the generated parser. For now, only some of the helpers and filename are generated based on this new option, but this will change in the future most likely. Resolves #421
7 years ago
/* Features that should be generated in the parser. */
const features = options.features || {};
function use( feature, use ) {
return feature in features
? !! features[ feature ]
: use == null
? true
: !! use;
}
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
/* These only indent non-empty lines to avoid trailing whitespace. */
const lineMatchRE = /^([^`\r\n]+?(?:`[^`]*?`[^\r\n]*?)?)$/gm;
function indent2( code ) {
return code.replace( lineMatchRE, " $1" );
Implement basic support for tracing Parsers can now be generated with support for tracing using the --trace CLI option or a boolean |trace| option to |PEG.buildParser|. This makes them trace their progress, which can be useful for debugging. Parsers generated with tracing support are called "tracing parsers". When a tracing parser executes, by default it traces the rules it enters and exits by writing messages to the console. For example, a parser built from this grammar: start = a / b a = "a" b = "b" will write this to the console when parsing input "b": 1:1 rule.enter start 1:1 rule.enter a 1:1 rule.fail a 1:1 rule.enter b 1:2 rule.match b 1:2 rule.match start You can customize tracing by passing a custom *tracer* to parser's |parse| method using the |tracer| option: parser.parse(input, { trace: tracer }); This will replace the built-in default tracer (which writes to the console) by the tracer you supplied. The tracer must be an object with a |trace| method. This method is called each time a tracing event happens. It takes one argument which is an object describing the tracing event. Currently, three events are supported: * rule.enter -- triggered when a rule is entered * rule.match -- triggered when a rule matches successfully * rule.fail -- triggered when a rule fails to match These events are triggered in nested pairs -- for each rule.enter event there is a matching rule.match or rule.fail event. The event object passed as an argument to |trace| contains these properties: * type -- event type * rule -- name of the rule the event is related to * offset -- parse position at the time of the event * line -- line at the time of the event * column -- column at the time of the event * result -- rule's match result (only for rule.match event) The whole tracing API is somewhat experimental (which is why it isn't documented properly yet) and I expect it will evolve over time as experience is gained. The default tracer is also somewhat bare-bones. I hope that PEG.js user community will develop more sophisticated tracers over time and I'll be able to integrate their best ideas into the default tracer.
10 years ago
}
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
function indent10( code ) {
return code.replace( lineMatchRE, " $1" );
Implement basic support for tracing Parsers can now be generated with support for tracing using the --trace CLI option or a boolean |trace| option to |PEG.buildParser|. This makes them trace their progress, which can be useful for debugging. Parsers generated with tracing support are called "tracing parsers". When a tracing parser executes, by default it traces the rules it enters and exits by writing messages to the console. For example, a parser built from this grammar: start = a / b a = "a" b = "b" will write this to the console when parsing input "b": 1:1 rule.enter start 1:1 rule.enter a 1:1 rule.fail a 1:1 rule.enter b 1:2 rule.match b 1:2 rule.match start You can customize tracing by passing a custom *tracer* to parser's |parse| method using the |tracer| option: parser.parse(input, { trace: tracer }); This will replace the built-in default tracer (which writes to the console) by the tracer you supplied. The tracer must be an object with a |trace| method. This method is called each time a tracing event happens. It takes one argument which is an object describing the tracing event. Currently, three events are supported: * rule.enter -- triggered when a rule is entered * rule.match -- triggered when a rule matches successfully * rule.fail -- triggered when a rule fails to match These events are triggered in nested pairs -- for each rule.enter event there is a matching rule.match or rule.fail event. The event object passed as an argument to |trace| contains these properties: * type -- event type * rule -- name of the rule the event is related to * offset -- parse position at the time of the event * line -- line at the time of the event * column -- column at the time of the event * result -- rule's match result (only for rule.match event) The whole tracing API is somewhat experimental (which is why it isn't documented properly yet) and I expect it will evolve over time as experience is gained. The default tracer is also somewhat bare-bones. I hope that PEG.js user community will develop more sophisticated tracers over time and I'll be able to integrate their best ideas into the default tracer.
10 years ago
Rename and generalize |generateCache{Header,Footer}| Rename |generateCache{Header,Footer}| to |generateRule{Header,Footer}| and change their responsibility to generate overall header/footer of a rule function (when optimizing for speed) or the |peg$parseRule| function (when optimizing for speed). This creates a natural place where to generate tracing code (coming soon).
10 years ago
}
Implement basic support for tracing Parsers can now be generated with support for tracing using the --trace CLI option or a boolean |trace| option to |PEG.buildParser|. This makes them trace their progress, which can be useful for debugging. Parsers generated with tracing support are called "tracing parsers". When a tracing parser executes, by default it traces the rules it enters and exits by writing messages to the console. For example, a parser built from this grammar: start = a / b a = "a" b = "b" will write this to the console when parsing input "b": 1:1 rule.enter start 1:1 rule.enter a 1:1 rule.fail a 1:1 rule.enter b 1:2 rule.match b 1:2 rule.match start You can customize tracing by passing a custom *tracer* to parser's |parse| method using the |tracer| option: parser.parse(input, { trace: tracer }); This will replace the built-in default tracer (which writes to the console) by the tracer you supplied. The tracer must be an object with a |trace| method. This method is called each time a tracing event happens. It takes one argument which is an object describing the tracing event. Currently, three events are supported: * rule.enter -- triggered when a rule is entered * rule.match -- triggered when a rule matches successfully * rule.fail -- triggered when a rule fails to match These events are triggered in nested pairs -- for each rule.enter event there is a matching rule.match or rule.fail event. The event object passed as an argument to |trace| contains these properties: * type -- event type * rule -- name of the rule the event is related to * offset -- parse position at the time of the event * line -- line at the time of the event * column -- column at the time of the event * result -- rule's match result (only for rule.match event) The whole tracing API is somewhat experimental (which is why it isn't documented properly yet) and I expect it will evolve over time as experience is gained. The default tracer is also somewhat bare-bones. I hope that PEG.js user community will develop more sophisticated tracers over time and I'll be able to integrate their best ideas into the default tracer.
10 years ago
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
const l = i => "peg$c" + i; // |literals[i]| of the abstract machine
const r = i => "peg$r" + i; // |classes[i]| of the abstract machine
const e = i => "peg$e" + i; // |expectations[i]| of the abstract machine
const f = i => "peg$f" + i; // |actions[i]| of the abstract machine
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
function generateTables() {
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
function buildLiteral( literal ) {
return `"${ js.stringEscape( literal ) }"`;
}
function buildRegexp( cls ) {
return "/^["
+ ( cls.inverted ? "^" : "" )
+ cls.value
.map( part =>
( Array.isArray( part )
? js.regexpClassEscape( part[ 0 ] )
+ "-"
+ js.regexpClassEscape( part[ 1 ] )
: js.regexpClassEscape( part ) )
)
.join( "" )
+ "]/"
+ ( cls.ignoreCase ? "i" : "" );
}
function buildExpectation( e ) {
switch ( e.type ) {
case "rule":
return `peg$otherExpectation("${ js.stringEscape( e.value ) }")`;
Use error emitter (Closes #430)
7 years ago
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
case "literal":
return "peg$literalExpectation(\""
+ js.stringEscape( e.value )
+ "\", "
+ e.ignoreCase
+ ")";
Use error emitter (Closes #430)
7 years ago
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
case "class": {
const parts = e.value.map( part =>
( Array.isArray( part )
? `["${ js.stringEscape( part[ 0 ] ) }", "${ js.stringEscape( part[ 1 ] ) }"]`
: `"${ js.stringEscape( part ) }"` )
);
return "peg$classExpectation(["
+ parts.join( ", " ) + "], "
+ e.inverted + ", "
+ e.ignoreCase
+ ")";
}
Use error emitter (Closes #430)
7 years ago
case "any":
return "peg$anyExpectation()";
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
// istanbul ignore next
Use error emitter (Closes #430)
7 years ago
default:
session.fatal( `Unknown expectation type (${ JSON.stringify( e ) })` );
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
}
}
function buildFunc( f ) {
return `function(${ f.params.join( ", " ) }) {${ f.body }}`;
}
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
if ( options.optimize === "size" ) {
return [
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
"var peg$literals = [",
indent2( ast.literals.map( buildLiteral ).join( ",\n" ) ),
"];",
"var peg$regexps = [",
indent2( ast.classes.map( buildRegexp ).join( ",\n" ) ),
"];",
"var peg$expectations = [",
indent2( ast.expectations.map( buildExpectation ).join( ",\n" ) ),
"];",
"var peg$functions = [",
indent2( ast.functions.map( buildFunc ).join( ",\n" ) ),
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
"];",
"",
"var peg$bytecode = [",
indent2( ast.rules
.map( rule =>
`peg$decode("${
js.stringEscape( rule.bytecode
.map( b => String.fromCharCode( b + 32 ) )
.join( "" )
)
}")`
)
.join( ",\n" )
),
"];"
].join( "\n" );
Code generator rewrite This is a complete rewrite of the PEG.js code generator. Its goals are: 1. Allow optimizing the generated parser code for code size as well as for parsing speed. 2. Prepare ground for future optimizations and big features (like incremental parsing). 2. Replace the old template-based code-generation system with something more lightweight and flexible. 4. General code cleanup (structure, style, variable names, ...). New Architecture ---------------- The new code generator consists of two steps: * Bytecode generator -- produces bytecode for an abstract virtual machine * JavaScript generator -- produces JavaScript code based on the bytecode The abstract virtual machine is stack-based. Originally I wanted to make it register-based, but it turned out that all the code related to it would be more complex and the bytecode itself would be longer (because of explicit register specifications in instructions). The only downsides of the stack-based approach seem to be few small inefficiencies (see e.g. the |NIP| instruction), which seem to be insignificant. The new generator allows optimizing for parsing speed or code size (you can choose using the |optimize| option of the |PEG.buildParser| method or the --optimize/-o option on the command-line). When optimizing for size, the JavaScript generator emits the bytecode together with its constant table and a generic bytecode interpreter. Because the interpreter is small and the bytecode and constant table grow only slowly with size of the grammar, the resulting parser is also small. When optimizing for speed, the JavaScript generator just compiles the bytecode into JavaScript. The generated code is relatively efficient, so the resulting parser is fast. Internal Identifiers -------------------- As a small bonus, all internal identifiers visible to user code in the initializer, actions and predicates are prefixed by |peg$|. This lowers the chance that identifiers in user code will conflict with the ones from PEG.js. It also makes using any internals in user code ugly, which is a good thing. This solves GH-92. Performance ----------- The new code generator improved parsing speed and parser code size significantly. The generated parsers are now: * 39% faster when optimizing for speed * 69% smaller when optimizing for size (without minification) * 31% smaller when optimizing for size (with minification) (Parsing speed was measured using the |benchmark/run| script. Code size was measured by generating parsers for examples in the |examples| directory and adding up the file sizes. Minification was done by |uglify --ascii| in version 1.3.4.) Final Note ---------- This is just a beginning! The new code generator lays a foundation upon which many optimizations and improvements can (and will) be made. Stay tuned :-)
12 years ago
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
}
Rename and generalize |generateCache{Header,Footer}| Rename |generateCache{Header,Footer}| to |generateRule{Header,Footer}| and change their responsibility to generate overall header/footer of a rule function (when optimizing for speed) or the |peg$parseRule| function (when optimizing for speed). This creates a natural place where to generate tracing code (coming soon).
10 years ago
Some cosmetic changes - Remove unused '__slice' variable - Nicely indent a map/concat chain - 'forEach' instead of 'map' in emitted code
7 years ago
return ast.literals
.map( ( c, i ) => "var " + l( i ) + " = " + buildLiteral( c ) + ";" )
.concat( "", ast.classes.map(
( c, i ) => "var " + r( i ) + " = " + buildRegexp( c ) + ";"
) )
.concat( "", ast.expectations.map(
( c, i ) => "var " + e( i ) + " = " + buildExpectation( c ) + ";"
) )
.concat( "", ast.functions.map(
( c, i ) => "var " + f( i ) + " = " + buildFunc( c ) + ";"
) )
.join( "\n" );
Rename and generalize |generateCache{Header,Footer}| Rename |generateCache{Header,Footer}| to |generateRule{Header,Footer}| and change their responsibility to generate overall header/footer of a rule function (when optimizing for speed) or the |peg$parseRule| function (when optimizing for speed). This creates a natural place where to generate tracing code (coming soon).
10 years ago
Implement basic support for tracing Parsers can now be generated with support for tracing using the --trace CLI option or a boolean |trace| option to |PEG.buildParser|. This makes them trace their progress, which can be useful for debugging. Parsers generated with tracing support are called "tracing parsers". When a tracing parser executes, by default it traces the rules it enters and exits by writing messages to the console. For example, a parser built from this grammar: start = a / b a = "a" b = "b" will write this to the console when parsing input "b": 1:1 rule.enter start 1:1 rule.enter a 1:1 rule.fail a 1:1 rule.enter b 1:2 rule.match b 1:2 rule.match start You can customize tracing by passing a custom *tracer* to parser's |parse| method using the |tracer| option: parser.parse(input, { trace: tracer }); This will replace the built-in default tracer (which writes to the console) by the tracer you supplied. The tracer must be an object with a |trace| method. This method is called each time a tracing event happens. It takes one argument which is an object describing the tracing event. Currently, three events are supported: * rule.enter -- triggered when a rule is entered * rule.match -- triggered when a rule matches successfully * rule.fail -- triggered when a rule fails to match These events are triggered in nested pairs -- for each rule.enter event there is a matching rule.match or rule.fail event. The event object passed as an argument to |trace| contains these properties: * type -- event type * rule -- name of the rule the event is related to * offset -- parse position at the time of the event * line -- line at the time of the event * column -- column at the time of the event * result -- rule's match result (only for rule.match event) The whole tracing API is somewhat experimental (which is why it isn't documented properly yet) and I expect it will evolve over time as experience is gained. The default tracer is also somewhat bare-bones. I hope that PEG.js user community will develop more sophisticated tracers over time and I'll be able to integrate their best ideas into the default tracer.
10 years ago
}
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
function generateRuleHeader( ruleNameCode, ruleIndexCode ) {
const parts = [];
Report consistent errors on look ahead + cached results This should resolve issue #452, and is based entirely on a fix @nikku did on a local branch of PEG.js v0.10.0 (Currently at https://github.com/nikku/pegjs/tree/452-peg-js-0.10-fix). Also his test case from #555 is included. Fixes #452, Closes #555
7 years ago
parts.push( [
"",
Restore silentFails guard for rule$expects
7 years ago
"var rule$expects = function (expected) {",
" if (peg$silentFails === 0) peg$expect(expected);",
"}",
Report consistent errors on look ahead + cached results This should resolve issue #452, and is based entirely on a fix @nikku did on a local branch of PEG.js v0.10.0 (Currently at https://github.com/nikku/pegjs/tree/452-peg-js-0.10-fix). Also his test case from #555 is included. Fixes #452, Closes #555
7 years ago
""
].join( "\n" ) );
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
if ( options.trace ) {
parts.push( [
"peg$tracer.trace({",
" type: \"rule.enter\",",
" rule: " + ruleNameCode + ",",
" location: peg$computeLocation(startPos, startPos)",
"});",
""
].join( "\n" ) );
}
if ( options.cache ) {
parts.push( [
"var key = peg$currPos * " + ast.rules.length + " + " + ruleIndexCode + ";",
"var cached = peg$resultsCache[key];",
Report consistent errors on look ahead + cached results This should resolve issue #452, and is based entirely on a fix @nikku did on a local branch of PEG.js v0.10.0 (Currently at https://github.com/nikku/pegjs/tree/452-peg-js-0.10-fix). Also his test case from #555 is included. Fixes #452, Closes #555
7 years ago
"var rule$expectations = [];",
"",
"rule$expects = function (expected) {",
" if (peg$silentFails === 0) peg$expect(expected);",
" rule$expectations.push(expected);",
"}",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
"",
"if (cached) {",
" peg$currPos = cached.nextPos;",
Report consistent errors on look ahead + cached results This should resolve issue #452, and is based entirely on a fix @nikku did on a local branch of PEG.js v0.10.0 (Currently at https://github.com/nikku/pegjs/tree/452-peg-js-0.10-fix). Also his test case from #555 is included. Fixes #452, Closes #555
7 years ago
"",
" rule$expectations = cached.expectations;",
" if (peg$silentFails === 0) {",
Some cosmetic changes - Remove unused '__slice' variable - Nicely indent a map/concat chain - 'forEach' instead of 'map' in emitted code
7 years ago
" rule$expectations.forEach(peg$expect);",
Report consistent errors on look ahead + cached results This should resolve issue #452, and is based entirely on a fix @nikku did on a local branch of PEG.js v0.10.0 (Currently at https://github.com/nikku/pegjs/tree/452-peg-js-0.10-fix). Also his test case from #555 is included. Fixes #452, Closes #555
7 years ago
" }",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
""
].join( "\n" ) );
if ( options.trace ) {
parts.push( [
"if (cached.result !== peg$FAILED) {",
" peg$tracer.trace({",
" type: \"rule.match\",",
" rule: " + ruleNameCode + ",",
" result: cached.result,",
" location: peg$computeLocation(startPos, peg$currPos)",
" });",
"} else {",
" peg$tracer.trace({",
" type: \"rule.fail\",",
" rule: " + ruleNameCode + ",",
" location: peg$computeLocation(startPos, startPos)",
" });",
"}",
""
].join( "\n" ) );
}
parts.push( [
" return cached.result;",
"}",
""
].join( "\n" ) );
}
return parts.join( "\n" );
Code generator rewrite This is a complete rewrite of the PEG.js code generator. Its goals are: 1. Allow optimizing the generated parser code for code size as well as for parsing speed. 2. Prepare ground for future optimizations and big features (like incremental parsing). 2. Replace the old template-based code-generation system with something more lightweight and flexible. 4. General code cleanup (structure, style, variable names, ...). New Architecture ---------------- The new code generator consists of two steps: * Bytecode generator -- produces bytecode for an abstract virtual machine * JavaScript generator -- produces JavaScript code based on the bytecode The abstract virtual machine is stack-based. Originally I wanted to make it register-based, but it turned out that all the code related to it would be more complex and the bytecode itself would be longer (because of explicit register specifications in instructions). The only downsides of the stack-based approach seem to be few small inefficiencies (see e.g. the |NIP| instruction), which seem to be insignificant. The new generator allows optimizing for parsing speed or code size (you can choose using the |optimize| option of the |PEG.buildParser| method or the --optimize/-o option on the command-line). When optimizing for size, the JavaScript generator emits the bytecode together with its constant table and a generic bytecode interpreter. Because the interpreter is small and the bytecode and constant table grow only slowly with size of the grammar, the resulting parser is also small. When optimizing for speed, the JavaScript generator just compiles the bytecode into JavaScript. The generated code is relatively efficient, so the resulting parser is fast. Internal Identifiers -------------------- As a small bonus, all internal identifiers visible to user code in the initializer, actions and predicates are prefixed by |peg$|. This lowers the chance that identifiers in user code will conflict with the ones from PEG.js. It also makes using any internals in user code ugly, which is a good thing. This solves GH-92. Performance ----------- The new code generator improved parsing speed and parser code size significantly. The generated parsers are now: * 39% faster when optimizing for speed * 69% smaller when optimizing for size (without minification) * 31% smaller when optimizing for size (with minification) (Parsing speed was measured using the |benchmark/run| script. Code size was measured by generating parsers for examples in the |examples| directory and adding up the file sizes. Minification was done by |uglify --ascii| in version 1.3.4.) Final Note ---------- This is just a beginning! The new code generator lays a foundation upon which many optimizations and improvements can (and will) be made. Stay tuned :-)
12 years ago
}
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
function generateRuleFooter( ruleNameCode, resultCode ) {
const parts = [];
if ( options.cache ) {
parts.push( [
"",
Report consistent errors on look ahead + cached results This should resolve issue #452, and is based entirely on a fix @nikku did on a local branch of PEG.js v0.10.0 (Currently at https://github.com/nikku/pegjs/tree/452-peg-js-0.10-fix). Also his test case from #555 is included. Fixes #452, Closes #555
7 years ago
"peg$resultsCache[key] = {",
" nextPos: peg$currPos,",
" result: " + resultCode + ",",
" expectations: rule$expectations",
"};"
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
].join( "\n" ) );
}
if ( options.trace ) {
parts.push( [
"",
"if (" + resultCode + " !== peg$FAILED) {",
" peg$tracer.trace({",
" type: \"rule.match\",",
" rule: " + ruleNameCode + ",",
" result: " + resultCode + ",",
" location: peg$computeLocation(startPos, peg$currPos)",
" });",
"} else {",
" peg$tracer.trace({",
" type: \"rule.fail\",",
" rule: " + ruleNameCode + ",",
" location: peg$computeLocation(startPos, startPos)",
" });",
"}"
].join( "\n" ) );
}
parts.push( [
"",
"return " + resultCode + ";"
].join( "\n" ) );
return parts.join( "\n" );
Code generator rewrite This is a complete rewrite of the PEG.js code generator. Its goals are: 1. Allow optimizing the generated parser code for code size as well as for parsing speed. 2. Prepare ground for future optimizations and big features (like incremental parsing). 2. Replace the old template-based code-generation system with something more lightweight and flexible. 4. General code cleanup (structure, style, variable names, ...). New Architecture ---------------- The new code generator consists of two steps: * Bytecode generator -- produces bytecode for an abstract virtual machine * JavaScript generator -- produces JavaScript code based on the bytecode The abstract virtual machine is stack-based. Originally I wanted to make it register-based, but it turned out that all the code related to it would be more complex and the bytecode itself would be longer (because of explicit register specifications in instructions). The only downsides of the stack-based approach seem to be few small inefficiencies (see e.g. the |NIP| instruction), which seem to be insignificant. The new generator allows optimizing for parsing speed or code size (you can choose using the |optimize| option of the |PEG.buildParser| method or the --optimize/-o option on the command-line). When optimizing for size, the JavaScript generator emits the bytecode together with its constant table and a generic bytecode interpreter. Because the interpreter is small and the bytecode and constant table grow only slowly with size of the grammar, the resulting parser is also small. When optimizing for speed, the JavaScript generator just compiles the bytecode into JavaScript. The generated code is relatively efficient, so the resulting parser is fast. Internal Identifiers -------------------- As a small bonus, all internal identifiers visible to user code in the initializer, actions and predicates are prefixed by |peg$|. This lowers the chance that identifiers in user code will conflict with the ones from PEG.js. It also makes using any internals in user code ugly, which is a good thing. This solves GH-92. Performance ----------- The new code generator improved parsing speed and parser code size significantly. The generated parsers are now: * 39% faster when optimizing for speed * 69% smaller when optimizing for size (without minification) * 31% smaller when optimizing for size (with minification) (Parsing speed was measured using the |benchmark/run| script. Code size was measured by generating parsers for examples in the |examples| directory and adding up the file sizes. Minification was done by |uglify --ascii| in version 1.3.4.) Final Note ---------- This is just a beginning! The new code generator lays a foundation upon which many optimizations and improvements can (and will) be made. Stay tuned :-)
12 years ago
}
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
function generateInterpreter() {
const parts = [];
function generateCondition( cond, argsLength ) {
const baseLength = argsLength + 3;
const thenLengthCode = "bc[ip + " + ( baseLength - 2 ) + "]";
const elseLengthCode = "bc[ip + " + ( baseLength - 1 ) + "]";
return [
"ends.push(end);",
"ips.push(ip + " + baseLength + " + " + thenLengthCode + " + " + elseLengthCode + ");",
"",
"if (" + cond + ") {",
" end = ip + " + baseLength + " + " + thenLengthCode + ";",
" ip += " + baseLength + ";",
"} else {",
" end = ip + " + baseLength + " + " + thenLengthCode + " + " + elseLengthCode + ";",
" ip += " + baseLength + " + " + thenLengthCode + ";",
"}",
"",
"break;"
].join( "\n" );
}
function generateLoop( cond ) {
const baseLength = 2;
const bodyLengthCode = "bc[ip + " + ( baseLength - 1 ) + "]";
return [
"if (" + cond + ") {",
" ends.push(end);",
" ips.push(ip);",
"",
" end = ip + " + baseLength + " + " + bodyLengthCode + ";",
" ip += " + baseLength + ";",
"} else {",
" ip += " + baseLength + " + " + bodyLengthCode + ";",
"}",
"",
"break;"
].join( "\n" );
}
function generateCall() {
const baseLength = 4;
const paramsLengthCode = "bc[ip + " + ( baseLength - 1 ) + "]";
return [
"params = bc.slice(ip + " + baseLength + ", ip + " + baseLength + " + " + paramsLengthCode + ")",
" .map(function(p) { return stack[stack.length - 1 - p]; });",
"",
"stack.splice(",
" stack.length - bc[ip + 2],",
" bc[ip + 2],",
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
" peg$functions[bc[ip + 1]].apply(null, params)",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
");",
"",
"ip += " + baseLength + " + " + paramsLengthCode + ";",
"break;"
].join( "\n" );
}
parts.push( [
"function peg$decode(s) {",
" return s.split(\"\").map(function(ch) { return ch.charCodeAt(0) - 32; });",
"}",
"",
"function peg$parseRule(index) {"
].join( "\n" ) );
if ( options.trace ) {
parts.push( [
" var bc = peg$bytecode[index];",
" var ip = 0;",
" var ips = [];",
" var end = bc.length;",
" var ends = [];",
" var stack = [];",
" var startPos = peg$currPos;",
" var params;"
].join( "\n" ) );
} else {
parts.push( [
" var bc = peg$bytecode[index];",
" var ip = 0;",
" var ips = [];",
" var end = bc.length;",
" var ends = [];",
" var stack = [];",
" var params;"
].join( "\n" ) );
}
parts.push( indent2( generateRuleHeader( "peg$ruleNames[index]", "index" ) ) );
parts.push( [
// The point of the outer loop and the |ips| & |ends| stacks is to avoid
// recursive calls for interpreting parts of bytecode. In other words, we
// implement the |interpret| operation of the abstract machine without
// function calls. Such calls would likely slow the parser down and more
// importantly cause stack overflows for complex grammars.
" while (true) {",
" while (ip < end) {",
" switch (bc[ip]) {",
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
" case " + op.PUSH_EMPTY_STRING + ":", // PUSH_EMPTY_STRING
" stack.push('');",
" ip++;",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
" break;",
"",
" case " + op.PUSH_UNDEFINED + ":", // PUSH_UNDEFINED
" stack.push(undefined);",
" ip++;",
" break;",
"",
" case " + op.PUSH_NULL + ":", // PUSH_NULL
" stack.push(null);",
" ip++;",
" break;",
"",
" case " + op.PUSH_FAILED + ":", // PUSH_FAILED
" stack.push(peg$FAILED);",
" ip++;",
" break;",
"",
" case " + op.PUSH_EMPTY_ARRAY + ":", // PUSH_EMPTY_ARRAY
" stack.push([]);",
" ip++;",
" break;",
"",
" case " + op.PUSH_CURR_POS + ":", // PUSH_CURR_POS
" stack.push(peg$currPos);",
" ip++;",
" break;",
"",
" case " + op.POP + ":", // POP
" stack.pop();",
" ip++;",
" break;",
"",
" case " + op.POP_CURR_POS + ":", // POP_CURR_POS
" peg$currPos = stack.pop();",
" ip++;",
" break;",
"",
" case " + op.POP_N + ":", // POP_N n
" stack.length -= bc[ip + 1];",
" ip += 2;",
" break;",
"",
" case " + op.NIP + ":", // NIP
" stack.splice(-2, 1);",
" ip++;",
" break;",
"",
" case " + op.APPEND + ":", // APPEND
" stack[stack.length - 2].push(stack.pop());",
" ip++;",
" break;",
"",
" case " + op.WRAP + ":", // WRAP n
" stack.push(stack.splice(stack.length - bc[ip + 1], bc[ip + 1]));",
" ip += 2;",
" break;",
"",
" case " + op.TEXT + ":", // TEXT
" stack.push(input.substring(stack.pop(), peg$currPos));",
" ip++;",
" break;",
"",
" case " + op.IF + ":", // IF t, f
indent10( generateCondition( "stack[stack.length - 1]", 0 ) ),
"",
" case " + op.IF_ERROR + ":", // IF_ERROR t, f
indent10( generateCondition(
"stack[stack.length - 1] === peg$FAILED",
0
) ),
"",
" case " + op.IF_NOT_ERROR + ":", // IF_NOT_ERROR t, f
indent10(
generateCondition( "stack[stack.length - 1] !== peg$FAILED",
0
) ),
"",
" case " + op.WHILE_NOT_ERROR + ":", // WHILE_NOT_ERROR b
indent10( generateLoop( "stack[stack.length - 1] !== peg$FAILED" ) ),
"",
" case " + op.MATCH_ANY + ":", // MATCH_ANY a, f, ...
indent10( generateCondition( "input.length > peg$currPos", 0 ) ),
"",
" case " + op.MATCH_STRING + ":", // MATCH_STRING s, a, f, ...
indent10( generateCondition(
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
"input.substr(peg$currPos, peg$literals[bc[ip + 1]].length) === peg$literals[bc[ip + 1]]",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
1
) ),
"",
" case " + op.MATCH_STRING_IC + ":", // MATCH_STRING_IC s, a, f, ...
indent10( generateCondition(
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
"input.substr(peg$currPos, peg$literals[bc[ip + 1]].length).toLowerCase() === peg$literals[bc[ip + 1]]",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
1
) ),
"",
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
" case " + op.MATCH_CLASS + ":", // MATCH_CLASS c, a, f, ...
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
indent10( generateCondition(
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
"peg$regexps[bc[ip + 1]].test(input.charAt(peg$currPos))",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
1
) ),
"",
" case " + op.ACCEPT_N + ":", // ACCEPT_N n
" stack.push(input.substr(peg$currPos, bc[ip + 1]));",
" peg$currPos += bc[ip + 1];",
" ip += 2;",
" break;",
"",
" case " + op.ACCEPT_STRING + ":", // ACCEPT_STRING s
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
" stack.push(peg$literals[bc[ip + 1]]);",
" peg$currPos += peg$literals[bc[ip + 1]].length;",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
" ip += 2;",
" break;",
"",
Partial fix #194 - nonsenses error messages with semantic predicates (#547) * Remove stack manipulations from FAIL opcode and rename FAIL to EXPECT * Always save the expected values regardless of the match result of expression * Remove unnecessary check for match result in the `named` node * Collect and process expectations from predicates
7 years ago
" case " + op.EXPECT + ":", // EXPECT e
Report consistent errors on look ahead + cached results This should resolve issue #452, and is based entirely on a fix @nikku did on a local branch of PEG.js v0.10.0 (Currently at https://github.com/nikku/pegjs/tree/452-peg-js-0.10-fix). Also his test case from #555 is included. Fixes #452, Closes #555
7 years ago
" rule$expects(peg$expectations[bc[ip + 1]]);",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
" ip += 2;",
" break;",
"",
" case " + op.LOAD_SAVED_POS + ":", // LOAD_SAVED_POS p
" peg$savedPos = stack[stack.length - 1 - bc[ip + 1]];",
" ip += 2;",
" break;",
"",
" case " + op.UPDATE_SAVED_POS + ":", // UPDATE_SAVED_POS
" peg$savedPos = peg$currPos;",
" ip++;",
" break;",
"",
" case " + op.CALL + ":", // CALL f, n, pc, p1, p2, ..., pN
indent10( generateCall() ),
"",
" case " + op.RULE + ":", // RULE r
" stack.push(peg$parseRule(bc[ip + 1]));",
" ip += 2;",
" break;",
"",
" case " + op.SILENT_FAILS_ON + ":", // SILENT_FAILS_ON
" peg$silentFails++;",
" ip++;",
" break;",
"",
" case " + op.SILENT_FAILS_OFF + ":", // SILENT_FAILS_OFF
" peg$silentFails--;",
" ip++;",
" break;",
"",
Partial fix #194 - nonsenses error messages with semantic predicates (#547) * Remove stack manipulations from FAIL opcode and rename FAIL to EXPECT * Always save the expected values regardless of the match result of expression * Remove unnecessary check for match result in the `named` node * Collect and process expectations from predicates
7 years ago
" case " + op.EXPECT_NS_BEGIN + ":", // EXPECT_NS_BEGIN
" peg$begin();",
" ip++;",
" break;",
"",
" case " + op.EXPECT_NS_END + ":", // EXPECT_NS_END invert
" peg$end(bc[ip + 1]);",
" ip += 2;",
" break;",
"",
Add code coverage by Intanbul
7 years ago
" // istanbul ignore next",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
" default:",
Add some useful debug information to some exceptions (#475) * Add some useful debug information to some exceptions * Add guard for visitor functions preventing from cryptic errors due to incomplete visitors * Add guard for js generator for preventing from cryptic errors due to incarrect stack manipulations
7 years ago
" throw new Error(",
" \"Rule #\" + index + \"" + ( options.trace ? " ('\" + peg$ruleNames[ index ] + \"')" : "" ) + ", position \" + ip + \": \"",
" + \"Invalid opcode \" + bc[ip] + \".\"",
" );",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
" }",
" }",
"",
" if (ends.length > 0) {",
" end = ends.pop();",
" ip = ips.pop();",
" } else {",
" break;",
" }",
" }"
].join( "\n" ) );
parts.push( indent2( generateRuleFooter( "peg$ruleNames[index]", "stack[0]" ) ) );
parts.push( "}" );
return parts.join( "\n" );
Code generator rewrite This is a complete rewrite of the PEG.js code generator. Its goals are: 1. Allow optimizing the generated parser code for code size as well as for parsing speed. 2. Prepare ground for future optimizations and big features (like incremental parsing). 2. Replace the old template-based code-generation system with something more lightweight and flexible. 4. General code cleanup (structure, style, variable names, ...). New Architecture ---------------- The new code generator consists of two steps: * Bytecode generator -- produces bytecode for an abstract virtual machine * JavaScript generator -- produces JavaScript code based on the bytecode The abstract virtual machine is stack-based. Originally I wanted to make it register-based, but it turned out that all the code related to it would be more complex and the bytecode itself would be longer (because of explicit register specifications in instructions). The only downsides of the stack-based approach seem to be few small inefficiencies (see e.g. the |NIP| instruction), which seem to be insignificant. The new generator allows optimizing for parsing speed or code size (you can choose using the |optimize| option of the |PEG.buildParser| method or the --optimize/-o option on the command-line). When optimizing for size, the JavaScript generator emits the bytecode together with its constant table and a generic bytecode interpreter. Because the interpreter is small and the bytecode and constant table grow only slowly with size of the grammar, the resulting parser is also small. When optimizing for speed, the JavaScript generator just compiles the bytecode into JavaScript. The generated code is relatively efficient, so the resulting parser is fast. Internal Identifiers -------------------- As a small bonus, all internal identifiers visible to user code in the initializer, actions and predicates are prefixed by |peg$|. This lowers the chance that identifiers in user code will conflict with the ones from PEG.js. It also makes using any internals in user code ugly, which is a good thing. This solves GH-92. Performance ----------- The new code generator improved parsing speed and parser code size significantly. The generated parsers are now: * 39% faster when optimizing for speed * 69% smaller when optimizing for size (without minification) * 31% smaller when optimizing for size (with minification) (Parsing speed was measured using the |benchmark/run| script. Code size was measured by generating parsers for examples in the |examples| directory and adding up the file sizes. Minification was done by |uglify --ascii| in version 1.3.4.) Final Note ---------- This is just a beginning! The new code generator lays a foundation upon which many optimizations and improvements can (and will) be made. Stay tuned :-)
12 years ago
}
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
function generateRuleFunction( rule ) {
const parts = [];
const stackVars = [];
function s( i ) {
Add some useful debug information to some exceptions (#475) * Add some useful debug information to some exceptions * Add guard for visitor functions preventing from cryptic errors due to incomplete visitors * Add guard for js generator for preventing from cryptic errors due to incarrect stack manipulations
7 years ago
// istanbul ignore next
Not session.fail but session.fatal
7 years ago
if ( i < 0 ) session.fatal( "Rule '" + rule.name + "': Var stack underflow: attempt to use var at index " + i );
Add some useful debug information to some exceptions (#475) * Add some useful debug information to some exceptions * Add guard for visitor functions preventing from cryptic errors due to incomplete visitors * Add guard for js generator for preventing from cryptic errors due to incarrect stack manipulations
7 years ago
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
return "s" + i;
} // |stack[i]| of the abstract machine
const stack = {
sp: -1,
maxSp: -1,
push( exprCode ) {
const code = s( ++this.sp ) + " = " + exprCode + ";";
if ( this.sp > this.maxSp ) this.maxSp = this.sp;
return code;
},
pop( n ) {
if ( typeof n === "undefined" ) return s( this.sp-- );
const values = Array( n );
for ( let i = 0; i < n; i++ ) {
values[ i ] = s( this.sp - n + 1 + i );
}
this.sp -= n;
return values;
},
top() {
return s( this.sp );
},
index( i ) {
return s( this.sp - i );
}
};
function compile( bc ) {
let ip = 0;
const end = bc.length;
const parts = [];
let value;
function compileCondition( cond, argCount ) {
Add some useful debug information to some exceptions (#475) * Add some useful debug information to some exceptions * Add guard for visitor functions preventing from cryptic errors due to incomplete visitors * Add guard for js generator for preventing from cryptic errors due to incarrect stack manipulations
7 years ago
const pos = ip;
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
const baseLength = argCount + 3;
const thenLength = bc[ ip + baseLength - 2 ];
const elseLength = bc[ ip + baseLength - 1 ];
const baseSp = stack.sp;
let thenCode, elseCode, thenSp, elseSp;
ip += baseLength;
thenCode = compile( bc.slice( ip, ip + thenLength ) );
thenSp = stack.sp;
ip += thenLength;
if ( elseLength > 0 ) {
stack.sp = baseSp;
elseCode = compile( bc.slice( ip, ip + elseLength ) );
elseSp = stack.sp;
ip += elseLength;
Add code coverage by Intanbul
7 years ago
// istanbul ignore if
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
if ( thenSp !== elseSp ) {
Not session.fail but session.fatal
7 years ago
session.fatal(
Add some useful debug information to some exceptions (#475) * Add some useful debug information to some exceptions * Add guard for visitor functions preventing from cryptic errors due to incomplete visitors * Add guard for js generator for preventing from cryptic errors due to incarrect stack manipulations
7 years ago
"Rule '" + rule.name + "', position " + pos + ": "
+ "Branches of a condition can't move the stack pointer differently "
+ "(before: " + baseSp + ", after then: " + thenSp + ", after else: " + elseSp + ")."
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
);
}
}
parts.push( "if (" + cond + ") {" );
parts.push( indent2( thenCode ) );
if ( elseLength > 0 ) {
parts.push( "} else {" );
parts.push( indent2( elseCode ) );
}
parts.push( "}" );
}
function compileLoop( cond ) {
Add some useful debug information to some exceptions (#475) * Add some useful debug information to some exceptions * Add guard for visitor functions preventing from cryptic errors due to incomplete visitors * Add guard for js generator for preventing from cryptic errors due to incarrect stack manipulations
7 years ago
const pos = ip;
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
const baseLength = 2;
const bodyLength = bc[ ip + baseLength - 1 ];
const baseSp = stack.sp;
let bodyCode, bodySp;
ip += baseLength;
bodyCode = compile( bc.slice( ip, ip + bodyLength ) );
bodySp = stack.sp;
ip += bodyLength;
Add code coverage by Intanbul
7 years ago
// istanbul ignore if
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
if ( bodySp !== baseSp ) {
Not session.fail but session.fatal
7 years ago
session.fatal(
Add some useful debug information to some exceptions (#475) * Add some useful debug information to some exceptions * Add guard for visitor functions preventing from cryptic errors due to incomplete visitors * Add guard for js generator for preventing from cryptic errors due to incarrect stack manipulations
7 years ago
"Rule '" + rule.name + "', position " + pos + ": "
+ "Body of a loop can't move the stack pointer "
+ "(before: " + baseSp + ", after: " + bodySp + ")."
);
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
}
parts.push( "while (" + cond + ") {" );
parts.push( indent2( bodyCode ) );
parts.push( "}" );
}
function compileCall() {
const baseLength = 4;
const paramsLength = bc[ ip + baseLength - 1 ];
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
const value = f( bc[ ip + 1 ] )
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
+ "("
+ bc
.slice( ip + baseLength, ip + baseLength + paramsLength )
.map( p => stack.index( p ) )
.join( ", " )
+ ")";
stack.pop( bc[ ip + 2 ] );
parts.push( stack.push( value ) );
ip += baseLength + paramsLength;
}
while ( ip < end ) {
switch ( bc[ ip ] ) {
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
case op.PUSH_EMPTY_STRING: // PUSH_EMPTY_STRING
parts.push( stack.push( "''" ) );
ip++;
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
break;
case op.PUSH_CURR_POS: // PUSH_CURR_POS
parts.push( stack.push( "peg$currPos" ) );
ip++;
break;
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
case op.PUSH_UNDEFINED: // PUSH_UNDEFINED
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
parts.push( stack.push( "undefined" ) );
ip++;
break;
case op.PUSH_NULL: // PUSH_NULL
parts.push( stack.push( "null" ) );
ip++;
break;
case op.PUSH_FAILED: // PUSH_FAILED
parts.push( stack.push( "peg$FAILED" ) );
ip++;
break;
case op.PUSH_EMPTY_ARRAY: // PUSH_EMPTY_ARRAY
parts.push( stack.push( "[]" ) );
ip++;
break;
case op.POP: // POP
stack.pop();
ip++;
break;
case op.POP_CURR_POS: // POP_CURR_POS
parts.push( "peg$currPos = " + stack.pop() + ";" );
ip++;
break;
case op.POP_N: // POP_N n
stack.pop( bc[ ip + 1 ] );
ip += 2;
break;
case op.NIP: // NIP
value = stack.pop();
stack.pop();
parts.push( stack.push( value ) );
ip++;
break;
case op.APPEND: // APPEND
value = stack.pop();
parts.push( stack.top() + ".push(" + value + ");" );
ip++;
break;
case op.WRAP: // WRAP n
parts.push(
stack.push( "[" + stack.pop( bc[ ip + 1 ] ).join( ", " ) + "]" )
);
ip += 2;
break;
case op.TEXT: // TEXT
parts.push(
stack.push( "input.substring(" + stack.pop() + ", peg$currPos)" )
);
ip++;
break;
case op.IF: // IF t, f
compileCondition( stack.top(), 0 );
break;
case op.IF_ERROR: // IF_ERROR t, f
compileCondition( stack.top() + " === peg$FAILED", 0 );
break;
case op.IF_NOT_ERROR: // IF_NOT_ERROR t, f
compileCondition( stack.top() + " !== peg$FAILED", 0 );
break;
case op.WHILE_NOT_ERROR: // WHILE_NOT_ERROR b
compileLoop( stack.top() + " !== peg$FAILED", 0 );
break;
case op.MATCH_ANY: // MATCH_ANY a, f, ...
compileCondition( "input.length > peg$currPos", 0 );
break;
case op.MATCH_STRING: // MATCH_STRING s, a, f, ...
compileCondition(
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
ast.literals[ bc[ ip + 1 ] ].length > 1
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
? "input.substr(peg$currPos, "
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
+ ast.literals[ bc[ ip + 1 ] ].length
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
+ ") === "
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
+ l( bc[ ip + 1 ] )
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
: "input.charCodeAt(peg$currPos) === "
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
+ ast.literals[ bc[ ip + 1 ] ].charCodeAt( 0 )
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
, 1
);
break;
case op.MATCH_STRING_IC: // MATCH_STRING_IC s, a, f, ...
compileCondition(
"input.substr(peg$currPos, "
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
+ ast.literals[ bc[ ip + 1 ] ].length
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
+ ").toLowerCase() === "
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
+ l( bc[ ip + 1 ] )
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
, 1
);
break;
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
case op.MATCH_CLASS: // MATCH_CLASS c, a, f, ...
compileCondition( r( bc[ ip + 1 ] ) + ".test(input.charAt(peg$currPos))", 1 );
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
break;
case op.ACCEPT_N: // ACCEPT_N n
parts.push( stack.push(
bc[ ip + 1 ] > 1
? "input.substr(peg$currPos, " + bc[ ip + 1 ] + ")"
: "input.charAt(peg$currPos)"
) );
parts.push(
bc[ ip + 1 ] > 1
? "peg$currPos += " + bc[ ip + 1 ] + ";"
: "peg$currPos++;"
);
ip += 2;
break;
case op.ACCEPT_STRING: // ACCEPT_STRING s
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
parts.push( stack.push( l( bc[ ip + 1 ] ) ) );
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
parts.push(
Move all codegeneration from `generateBytecode` pass to `generateJs` pass (#459) * Split 'consts' collection by content types into: - literals: for literal expressions, like `"a"` - classes: for character class expressions, like `[a]` - expectations: for constants describing expected values when parse failed - functions: for constants with function code * Move any JavaScript code generation from 'generateBytecode' to 'generateJs'. * Rename opcode 'MATCH_REGEXP' to 'MATCH_CLASS' (name reflects purpose, not implementation). * Replace 'PUSH' opcode with 'PUSH_EMPTY_STRING' opcode because it is only used with empty strings
7 years ago
ast.literals[ bc[ ip + 1 ] ].length > 1
? "peg$currPos += " + ast.literals[ bc[ ip + 1 ] ].length + ";"
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
: "peg$currPos++;"
);
ip += 2;
break;
Partial fix #194 - nonsenses error messages with semantic predicates (#547) * Remove stack manipulations from FAIL opcode and rename FAIL to EXPECT * Always save the expected values regardless of the match result of expression * Remove unnecessary check for match result in the `named` node * Collect and process expectations from predicates
7 years ago
case op.EXPECT: // EXPECT e
Report consistent errors on look ahead + cached results This should resolve issue #452, and is based entirely on a fix @nikku did on a local branch of PEG.js v0.10.0 (Currently at https://github.com/nikku/pegjs/tree/452-peg-js-0.10-fix). Also his test case from #555 is included. Fixes #452, Closes #555
7 years ago
parts.push( "rule$expects(" + e( bc[ ip + 1 ] ) + ");" );
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
ip += 2;
break;
case op.LOAD_SAVED_POS: // LOAD_SAVED_POS p
parts.push( "peg$savedPos = " + stack.index( bc[ ip + 1 ] ) + ";" );
ip += 2;
break;
case op.UPDATE_SAVED_POS: // UPDATE_SAVED_POS
parts.push( "peg$savedPos = peg$currPos;" );
ip++;
break;
case op.CALL: // CALL f, n, pc, p1, p2, ..., pN
compileCall();
break;
case op.RULE: // RULE r
parts.push( stack.push( "peg$parse" + ast.rules[ bc[ ip + 1 ] ].name + "()" ) );
ip += 2;
break;
case op.SILENT_FAILS_ON: // SILENT_FAILS_ON
parts.push( "peg$silentFails++;" );
ip++;
break;
case op.SILENT_FAILS_OFF: // SILENT_FAILS_OFF
parts.push( "peg$silentFails--;" );
ip++;
break;
Partial fix #194 - nonsenses error messages with semantic predicates (#547) * Remove stack manipulations from FAIL opcode and rename FAIL to EXPECT * Always save the expected values regardless of the match result of expression * Remove unnecessary check for match result in the `named` node * Collect and process expectations from predicates
7 years ago
case op.EXPECT_NS_BEGIN: // EXPECT_NS_BEGIN
parts.push( "peg$begin();" );
ip++;
break;
case op.EXPECT_NS_END: // EXPECT_NS_END invert
parts.push( "peg$end(" + ( bc[ ip + 1 ] !== 0 ) + ");" );
ip += 2;
break;
Add code coverage by Intanbul
7 years ago
// istanbul ignore next
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
default:
Not session.fail but session.fatal
7 years ago
session.fatal(
Add some useful debug information to some exceptions (#475) * Add some useful debug information to some exceptions * Add guard for visitor functions preventing from cryptic errors due to incomplete visitors * Add guard for js generator for preventing from cryptic errors due to incarrect stack manipulations
7 years ago
"Rule '" + rule.name + "', position " + ip + ": "
+ "Invalid opcode " + bc[ ip ] + "."
);
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
}
}
return parts.join( "\n" );
}
const code = compile( rule.bytecode );
parts.push( "function peg$parse" + rule.name + "() {" );
if ( options.trace ) {
parts.push( " var startPos = peg$currPos;" );
}
for ( let i = 0; i <= stack.maxSp; i++ ) {
stackVars[ i ] = s( i );
}
parts.push( " var " + stackVars.join( ", " ) + ";" );
parts.push( indent2( generateRuleHeader(
"\"" + js.stringEscape( rule.name ) + "\"",
Merge ast utils into Grammar class
7 years ago
ast.indexOfRule( rule.name )
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
) ) );
parts.push( indent2( code ) );
parts.push( indent2( generateRuleFooter(
"\"" + js.stringEscape( rule.name ) + "\"",
s( 0 )
) ) );
parts.push( "}" );
return parts.join( "\n" );
Improve location info in tracing events Replace |line|, |column|, and |offset| properties of tracing events with the |location| property. It contains an object similar to the one returned by the |location| function available in action code: { start: { offset: 23, line: 5, column: 6 }, end: { offset: 25, line: 5, column: 8 } } For the |rule.match| event, |start| refers to the position at the beginning of the matched input and |end| refers to the position after the end of the matched input. For |rule.enter| and |rule.fail| events, both |start| and |end| refer to the current position at the time the rule was entered.
10 years ago
}
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
function generateToplevel() {
const parts = [];
parts.push( [
"function peg$subclass(child, parent) {",
" function C() { this.constructor = child; }",
" C.prototype = parent.prototype;",
" child.prototype = new C();",
"}",
"",
"function peg$SyntaxError(message, expected, found, location) {",
" this.message = message;",
" this.expected = expected;",
" this.found = found;",
" this.location = location;",
" this.name = \"SyntaxError\";",
"",
Add code coverage by Intanbul
7 years ago
" // istanbul ignore next",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
" if (typeof Error.captureStackTrace === \"function\") {",
" Error.captureStackTrace(this, peg$SyntaxError);",
" }",
"}",
"",
"peg$subclass(peg$SyntaxError, Error);",
"",
"peg$SyntaxError.buildMessage = function(expected, found) {",
" var DESCRIBE_EXPECTATION_FNS = {",
" literal: function(expectation) {",
" return \"\\\"\" + literalEscape(expectation.text) + \"\\\"\";",
" },",
"",
" class: function(expectation) {",
" var escapedParts = expectation.parts.map(function(part) {",
" return Array.isArray(part)",
" ? classEscape(part[0]) + \"-\" + classEscape(part[1])",
" : classEscape(part);",
" });",
"",
" return \"[\" + (expectation.inverted ? \"^\" : \"\") + escapedParts + \"]\";",
" },",
"",
" any: function() {",
" return \"any character\";",
" },",
"",
" end: function() {",
" return \"end of input\";",
" },",
"",
" other: function(expectation) {",
" return expectation.description;",
Partial fix #194 - nonsenses error messages with semantic predicates (#547) * Remove stack manipulations from FAIL opcode and rename FAIL to EXPECT * Always save the expected values regardless of the match result of expression * Remove unnecessary check for match result in the `named` node * Collect and process expectations from predicates
7 years ago
" },",
"",
" not: function(expectation) {",
" return \"not \" + describeExpectation(expectation.expected);",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
" }",
" };",
"",
" function hex(ch) {",
" return ch.charCodeAt(0).toString(16).toUpperCase();",
" }",
"",
" function literalEscape(s) {",
" return s",
" .replace(/\\\\/g, \"\\\\\\\\\")", // backslash
" .replace(/\"/g, \"\\\\\\\"\")", // closing double quote
" .replace(/\\0/g, \"\\\\0\")", // null
" .replace(/\\t/g, \"\\\\t\")", // horizontal tab
" .replace(/\\n/g, \"\\\\n\")", // line feed
" .replace(/\\r/g, \"\\\\r\")", // carriage return
" .replace(/[\\x00-\\x0F]/g, function(ch) { return \"\\\\x0\" + hex(ch); })",
" .replace(/[\\x10-\\x1F\\x7F-\\x9F]/g, function(ch) { return \"\\\\x\" + hex(ch); });",
" }",
"",
" function classEscape(s) {",
" return s",
" .replace(/\\\\/g, \"\\\\\\\\\")", // backslash
" .replace(/\\]/g, \"\\\\]\")", // closing bracket
" .replace(/\\^/g, \"\\\\^\")", // caret
" .replace(/-/g, \"\\\\-\")", // dash
" .replace(/\\0/g, \"\\\\0\")", // null
" .replace(/\\t/g, \"\\\\t\")", // horizontal tab
" .replace(/\\n/g, \"\\\\n\")", // line feed
" .replace(/\\r/g, \"\\\\r\")", // carriage return
" .replace(/[\\x00-\\x0F]/g, function(ch) { return \"\\\\x0\" + hex(ch); })",
" .replace(/[\\x10-\\x1F\\x7F-\\x9F]/g, function(ch) { return \"\\\\x\" + hex(ch); });",
" }",
"",
" function describeExpectation(expectation) {",
" return DESCRIBE_EXPECTATION_FNS[expectation.type](expectation);",
" }",
"",
" function describeExpected(expected) {",
" var descriptions = expected.map(describeExpectation);",
" var i, j;",
"",
" descriptions.sort();",
"",
" if (descriptions.length > 0) {",
" for (i = 1, j = 1; i < descriptions.length; i++) {",
" if (descriptions[i - 1] !== descriptions[i]) {",
" descriptions[j] = descriptions[i];",
" j++;",
" }",
" }",
" descriptions.length = j;",
" }",
"",
" switch (descriptions.length) {",
" case 1:",
" return descriptions[0];",
"",
" case 2:",
" return descriptions[0] + \" or \" + descriptions[1];",
"",
" default:",
" return descriptions.slice(0, -1).join(\", \")",
" + \", or \"",
" + descriptions[descriptions.length - 1];",
" }",
" }",
"",
" function describeFound(found) {",
" return found ? \"\\\"\" + literalEscape(found) + \"\\\"\" : \"end of input\";",
" }",
"",
" return \"Expected \" + describeExpected(expected) + \" but \" + describeFound(found) + \" found.\";",
"};",
""
].join( "\n" ) );
if ( options.trace ) {
parts.push( [
"function peg$DefaultTracer() {",
" this.indentLevel = 0;",
"}",
"",
"peg$DefaultTracer.prototype.trace = function(event) {",
" var that = this;",
"",
" function log(event) {",
" function repeat(string, n) {",
" var result = \"\", i;",
"",
" for (i = 0; i < n; i++) {",
" result += string;",
" }",
"",
" return result;",
" }",
"",
" function pad(string, length) {",
" return string + repeat(\" \", length - string.length);",
" }",
"",
" if (typeof console === \"object\") {", // IE 8-10
" console.log(",
" event.location.start.line + \":\" + event.location.start.column + \"-\"",
" + event.location.end.line + \":\" + event.location.end.column + \" \"",
" + pad(event.type, 10) + \" \"",
" + repeat(\" \", that.indentLevel) + event.rule",
" );",
" }",
" }",
"",
" switch (event.type) {",
" case \"rule.enter\":",
" log(event);",
" this.indentLevel++;",
" break;",
"",
" case \"rule.match\":",
" this.indentLevel--;",
" log(event);",
" break;",
"",
" case \"rule.fail\":",
" this.indentLevel--;",
" log(event);",
" break;",
"",
Add code coverage by Intanbul
7 years ago
" // istanbul ignore next",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
" default:",
" throw new Error(\"Invalid event type: \" + event.type + \".\");",
" }",
"};",
""
].join( "\n" ) );
}
parts.push( [
"function peg$parse(input, options) {",
" options = options !== undefined ? options : {};",
"",
" var peg$FAILED = {};",
""
].join( "\n" ) );
if ( options.optimize === "size" ) {
const startRuleIndices = "{ "
+ options.allowedStartRules
Merge ast utils into Grammar class
7 years ago
.map( r => r + ": " + ast.indexOfRule( r ) )
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
.join( ", " )
+ " }";
Merge ast utils into Grammar class
7 years ago
const startRuleIndex = ast.indexOfRule( options.allowedStartRules[ 0 ] );
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
parts.push( [
" var peg$startRuleIndices = " + startRuleIndices + ";",
" var peg$startRuleIndex = " + startRuleIndex + ";"
].join( "\n" ) );
Improve indentation of variable declarations Before this commit, continuation lines of multi-line values in variable declaration initializers were aligned with the variable name: let foo = { a: 5, b: 6 }; This was highly irregular, maintenance intensive, and made declarations look different from assignments. This commit changes the indentation to be more regular and similar to assignments: let foo = { a: 5, b: 6 };
8 years ago
} else {
Don't use "arrays" in lib/compiler/passes/generate-js.js The "arrays" module will be removed. See #442.
8 years ago
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
const startRuleFunctions = "{ "
+ options.allowedStartRules
.map( r => r + ": peg$parse" + r )
.join( ", " )
+ " }";
const startRuleFunction = "peg$parse" + options.allowedStartRules[ 0 ];
Don't use "arrays" in lib/compiler/passes/generate-js.js The "arrays" module will be removed. See #442.
8 years ago
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
parts.push( [
" var peg$startRuleFunctions = " + startRuleFunctions + ";",
" var peg$startRuleFunction = " + startRuleFunction + ";"
].join( "\n" ) );
Code generator rewrite This is a complete rewrite of the PEG.js code generator. Its goals are: 1. Allow optimizing the generated parser code for code size as well as for parsing speed. 2. Prepare ground for future optimizations and big features (like incremental parsing). 2. Replace the old template-based code-generation system with something more lightweight and flexible. 4. General code cleanup (structure, style, variable names, ...). New Architecture ---------------- The new code generator consists of two steps: * Bytecode generator -- produces bytecode for an abstract virtual machine * JavaScript generator -- produces JavaScript code based on the bytecode The abstract virtual machine is stack-based. Originally I wanted to make it register-based, but it turned out that all the code related to it would be more complex and the bytecode itself would be longer (because of explicit register specifications in instructions). The only downsides of the stack-based approach seem to be few small inefficiencies (see e.g. the |NIP| instruction), which seem to be insignificant. The new generator allows optimizing for parsing speed or code size (you can choose using the |optimize| option of the |PEG.buildParser| method or the --optimize/-o option on the command-line). When optimizing for size, the JavaScript generator emits the bytecode together with its constant table and a generic bytecode interpreter. Because the interpreter is small and the bytecode and constant table grow only slowly with size of the grammar, the resulting parser is also small. When optimizing for speed, the JavaScript generator just compiles the bytecode into JavaScript. The generated code is relatively efficient, so the resulting parser is fast. Internal Identifiers -------------------- As a small bonus, all internal identifiers visible to user code in the initializer, actions and predicates are prefixed by |peg$|. This lowers the chance that identifiers in user code will conflict with the ones from PEG.js. It also makes using any internals in user code ugly, which is a good thing. This solves GH-92. Performance ----------- The new code generator improved parsing speed and parser code size significantly. The generated parsers are now: * 39% faster when optimizing for speed * 69% smaller when optimizing for size (without minification) * 31% smaller when optimizing for size (with minification) (Parsing speed was measured using the |benchmark/run| script. Code size was measured by generating parsers for examples in the |examples| directory and adding up the file sizes. Minification was done by |uglify --ascii| in version 1.3.4.) Final Note ---------- This is just a beginning! The new code generator lays a foundation upon which many optimizations and improvements can (and will) be made. Stay tuned :-)
12 years ago
}
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
parts.push( "" );
parts.push( indent2( generateTables() ) );
parts.push( [
"",
" var peg$currPos = 0;",
" var peg$savedPos = 0;",
" var peg$posDetailsCache = [{ line: 1, column: 1 }];",
Partial fix #194 - nonsenses error messages with semantic predicates (#547) * Remove stack manipulations from FAIL opcode and rename FAIL to EXPECT * Always save the expected values regardless of the match result of expression * Remove unnecessary check for match result in the `named` node * Collect and process expectations from predicates
7 years ago
" var peg$expected = [];",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
" var peg$silentFails = 0;", // 0 = report failures, > 0 = silence failures
""
].join( "\n" ) );
if ( options.cache ) {
parts.push( [
" var peg$resultsCache = {};",
""
].join( "\n" ) );
Code generator rewrite This is a complete rewrite of the PEG.js code generator. Its goals are: 1. Allow optimizing the generated parser code for code size as well as for parsing speed. 2. Prepare ground for future optimizations and big features (like incremental parsing). 2. Replace the old template-based code-generation system with something more lightweight and flexible. 4. General code cleanup (structure, style, variable names, ...). New Architecture ---------------- The new code generator consists of two steps: * Bytecode generator -- produces bytecode for an abstract virtual machine * JavaScript generator -- produces JavaScript code based on the bytecode The abstract virtual machine is stack-based. Originally I wanted to make it register-based, but it turned out that all the code related to it would be more complex and the bytecode itself would be longer (because of explicit register specifications in instructions). The only downsides of the stack-based approach seem to be few small inefficiencies (see e.g. the |NIP| instruction), which seem to be insignificant. The new generator allows optimizing for parsing speed or code size (you can choose using the |optimize| option of the |PEG.buildParser| method or the --optimize/-o option on the command-line). When optimizing for size, the JavaScript generator emits the bytecode together with its constant table and a generic bytecode interpreter. Because the interpreter is small and the bytecode and constant table grow only slowly with size of the grammar, the resulting parser is also small. When optimizing for speed, the JavaScript generator just compiles the bytecode into JavaScript. The generated code is relatively efficient, so the resulting parser is fast. Internal Identifiers -------------------- As a small bonus, all internal identifiers visible to user code in the initializer, actions and predicates are prefixed by |peg$|. This lowers the chance that identifiers in user code will conflict with the ones from PEG.js. It also makes using any internals in user code ugly, which is a good thing. This solves GH-92. Performance ----------- The new code generator improved parsing speed and parser code size significantly. The generated parsers are now: * 39% faster when optimizing for speed * 69% smaller when optimizing for size (without minification) * 31% smaller when optimizing for size (with minification) (Parsing speed was measured using the |benchmark/run| script. Code size was measured by generating parsers for examples in the |examples| directory and adding up the file sizes. Minification was done by |uglify --ascii| in version 1.3.4.) Final Note ---------- This is just a beginning! The new code generator lays a foundation upon which many optimizations and improvements can (and will) be made. Stay tuned :-)
12 years ago
}
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
if ( options.trace ) {
if ( options.optimize === "size" ) {
const ruleNames = "["
+ ast.rules
.map( r => `"${ js.stringEscape( r.name ) }"` )
.join( ", " )
+ "]";
parts.push( [
" var peg$ruleNames = " + ruleNames + ";",
""
].join( "\n" ) );
}
parts.push( [
" var peg$tracer = \"tracer\" in options ? options.tracer : new peg$DefaultTracer();",
""
].join( "\n" ) );
Assert that generated bytecode manipulates stack correctly There are two invariants in generated bytecode related to the stack: 1. Branches of a condition must move the stack pointer in the same way. 2. Body of a loop can't move the stack pointer. These invariants were always true, but they were not checked. Now we check them at least when compiling with optimization for speed, because there we analyze the stack pointer movements statically.
11 years ago
}
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
parts.push( [
" var peg$result;",
""
].join( "\n" ) );
if ( options.optimize === "size" ) {
parts.push( [
" if (\"startRule\" in options) {",
" if (!(options.startRule in peg$startRuleIndices)) {",
" throw new Error(\"Can't start parsing from rule \\\"\" + options.startRule + \"\\\".\");",
" }",
"",
" peg$startRuleIndex = peg$startRuleIndices[options.startRule];",
" }"
].join( "\n" ) );
} else {
parts.push( [
" if (\"startRule\" in options) {",
" if (!(options.startRule in peg$startRuleFunctions)) {",
" throw new Error(\"Can't start parsing from rule \\\"\" + options.startRule + \"\\\".\");",
" }",
"",
" peg$startRuleFunction = peg$startRuleFunctions[options.startRule];",
" }"
].join( "\n" ) );
Code generator rewrite This is a complete rewrite of the PEG.js code generator. Its goals are: 1. Allow optimizing the generated parser code for code size as well as for parsing speed. 2. Prepare ground for future optimizations and big features (like incremental parsing). 2. Replace the old template-based code-generation system with something more lightweight and flexible. 4. General code cleanup (structure, style, variable names, ...). New Architecture ---------------- The new code generator consists of two steps: * Bytecode generator -- produces bytecode for an abstract virtual machine * JavaScript generator -- produces JavaScript code based on the bytecode The abstract virtual machine is stack-based. Originally I wanted to make it register-based, but it turned out that all the code related to it would be more complex and the bytecode itself would be longer (because of explicit register specifications in instructions). The only downsides of the stack-based approach seem to be few small inefficiencies (see e.g. the |NIP| instruction), which seem to be insignificant. The new generator allows optimizing for parsing speed or code size (you can choose using the |optimize| option of the |PEG.buildParser| method or the --optimize/-o option on the command-line). When optimizing for size, the JavaScript generator emits the bytecode together with its constant table and a generic bytecode interpreter. Because the interpreter is small and the bytecode and constant table grow only slowly with size of the grammar, the resulting parser is also small. When optimizing for speed, the JavaScript generator just compiles the bytecode into JavaScript. The generated code is relatively efficient, so the resulting parser is fast. Internal Identifiers -------------------- As a small bonus, all internal identifiers visible to user code in the initializer, actions and predicates are prefixed by |peg$|. This lowers the chance that identifiers in user code will conflict with the ones from PEG.js. It also makes using any internals in user code ugly, which is a good thing. This solves GH-92. Performance ----------- The new code generator improved parsing speed and parser code size significantly. The generated parsers are now: * 39% faster when optimizing for speed * 69% smaller when optimizing for size (without minification) * 31% smaller when optimizing for size (with minification) (Parsing speed was measured using the |benchmark/run| script. Code size was measured by generating parsers for examples in the |examples| directory and adding up the file sizes. Minification was done by |uglify --ascii| in version 1.3.4.) Final Note ---------- This is just a beginning! The new code generator lays a foundation upon which many optimizations and improvements can (and will) be made. Stay tuned :-)
12 years ago
}
Optional features This commit enables optional features that are enabled by default in the generated parser. For now, only some of the helpers and filename are generated based on this new option, but this will change in the future most likely. Resolves #421
7 years ago
if ( use( "text" ) ) {
parts.push( [
"",
" function text() {",
" return input.substring(peg$savedPos, peg$currPos);",
" }",
].join( "\n" ) );
}
if ( use( "offset" ) ) {
parts.push( [
"",
" function offset() {",
" return peg$savedPos;",
" }",
].join( "\n" ) );
}
if ( use( "range" ) ) {
parts.push( [
"",
" function range() {",
" return [peg$savedPos, peg$currPos];",
" }",
].join( "\n" ) );
}
if ( use( "location" ) ) {
parts.push( [
"",
" function location() {",
" return peg$computeLocation(peg$savedPos, peg$currPos);",
" }",
].join( "\n" ) );
}
if ( use( "expected" ) ) {
parts.push( [
"",
" function expected(description, location) {",
" location = location !== undefined",
" ? location",
" : peg$computeLocation(peg$savedPos, peg$currPos);",
"",
" throw peg$buildStructuredError(",
" [peg$otherExpectation(description)],",
" input.substring(peg$savedPos, peg$currPos),",
" location",
" );",
" }",
].join( "\n" ) );
}
if ( use( "error" ) ) {
parts.push( [
"",
" function error(message, location) {",
" location = location !== undefined",
" ? location",
" : peg$computeLocation(peg$savedPos, peg$currPos);",
"",
" throw peg$buildSimpleError(message, location);",
" }",
].join( "\n" ) );
}
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
parts.push( [
"",
" function peg$literalExpectation(text, ignoreCase) {",
" return { type: \"literal\", text: text, ignoreCase: ignoreCase };",
" }",
"",
" function peg$classExpectation(parts, inverted, ignoreCase) {",
" return { type: \"class\", parts: parts, inverted: inverted, ignoreCase: ignoreCase };",
" }",
"",
" function peg$anyExpectation() {",
" return { type: \"any\" };",
" }",
"",
" function peg$endExpectation() {",
" return { type: \"end\" };",
" }",
"",
" function peg$otherExpectation(description) {",
" return { type: \"other\", description: description };",
" }",
"",
" function peg$computePosDetails(pos) {",
" var details = peg$posDetailsCache[pos];",
" var p;",
"",
" if (details) {",
" return details;",
" } else {",
" p = pos - 1;",
" while (!peg$posDetailsCache[p]) {",
" p--;",
" }",
"",
" details = peg$posDetailsCache[p];",
" details = {",
" line: details.line,",
" column: details.column",
" };",
"",
" while (p < pos) {",
" if (input.charCodeAt(p) === 10) {",
" details.line++;",
" details.column = 1;",
" } else {",
" details.column++;",
" }",
"",
" p++;",
" }",
"",
" peg$posDetailsCache[pos] = details;",
"",
" return details;",
" }",
" }",
"",
Optional features This commit enables optional features that are enabled by default in the generated parser. For now, only some of the helpers and filename are generated based on this new option, but this will change in the future most likely. Resolves #421
7 years ago
use( "filename" ) ? " var peg$VALIDFILENAME = typeof options.filename === \"string\" && options.filename.length > 0;" : "",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
" function peg$computeLocation(startPos, endPos) {",
Optional features This commit enables optional features that are enabled by default in the generated parser. For now, only some of the helpers and filename are generated based on this new option, but this will change in the future most likely. Resolves #421
7 years ago
" var loc = {};",
"",
use( "filename" ) ? " if ( peg$VALIDFILENAME ) loc.filename = options.filename;" : "",
"",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
" var startPosDetails = peg$computePosDetails(startPos);",
Optional features This commit enables optional features that are enabled by default in the generated parser. For now, only some of the helpers and filename are generated based on this new option, but this will change in the future most likely. Resolves #421
7 years ago
" loc.start = {",
" offset: startPos,",
" line: startPosDetails.line,",
" column: startPosDetails.column",
" };",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
"",
Optional features This commit enables optional features that are enabled by default in the generated parser. For now, only some of the helpers and filename are generated based on this new option, but this will change in the future most likely. Resolves #421
7 years ago
" var endPosDetails = peg$computePosDetails(endPos);",
" loc.end = {",
" offset: endPos,",
" line: endPosDetails.line,",
" column: endPosDetails.column",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
" };",
Optional features This commit enables optional features that are enabled by default in the generated parser. For now, only some of the helpers and filename are generated based on this new option, but this will change in the future most likely. Resolves #421
7 years ago
"",
" return loc;",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
" }",
"",
Partial fix #194 - nonsenses error messages with semantic predicates (#547) * Remove stack manipulations from FAIL opcode and rename FAIL to EXPECT * Always save the expected values regardless of the match result of expression * Remove unnecessary check for match result in the `named` node * Collect and process expectations from predicates
7 years ago
" function peg$begin() {",
" peg$expected.push({ pos: peg$currPos, variants: [] });",
" }",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
"",
Partial fix #194 - nonsenses error messages with semantic predicates (#547) * Remove stack manipulations from FAIL opcode and rename FAIL to EXPECT * Always save the expected values regardless of the match result of expression * Remove unnecessary check for match result in the `named` node * Collect and process expectations from predicates
7 years ago
" function peg$expect(expected) {",
" var top = peg$expected[peg$expected.length - 1];",
"",
" if (peg$currPos < top.pos) { return; }",
"",
" if (peg$currPos > top.pos) {",
" top.pos = peg$currPos;",
" top.variants = [];",
" }",
"",
" top.variants.push(expected);",
" }",
"",
" function peg$end(invert) {",
" var expected = peg$expected.pop();",
" var top = peg$expected[peg$expected.length - 1];",
" var variants = expected.variants;",
"",
" if (top.pos !== expected.pos) { return; }",
"",
" if (invert) {",
" variants = variants.map(function(e) {",
" return e.type === \"not\" ? e.expected : { type: \"not\", expected: e };",
" });",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
" }",
"",
Partial fix #194 - nonsenses error messages with semantic predicates (#547) * Remove stack manipulations from FAIL opcode and rename FAIL to EXPECT * Always save the expected values regardless of the match result of expression * Remove unnecessary check for match result in the `named` node * Collect and process expectations from predicates
7 years ago
" Array.prototype.push.apply(top.variants, variants);",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
" }",
"",
" function peg$buildSimpleError(message, location) {",
" return new peg$SyntaxError(message, null, null, location);",
" }",
"",
" function peg$buildStructuredError(expected, found, location) {",
" return new peg$SyntaxError(",
" peg$SyntaxError.buildMessage(expected, found),",
" expected,",
" found,",
" location",
" );",
" }",
Partial fix #194 - nonsenses error messages with semantic predicates (#547) * Remove stack manipulations from FAIL opcode and rename FAIL to EXPECT * Always save the expected values regardless of the match result of expression * Remove unnecessary check for match result in the `named` node * Collect and process expectations from predicates
7 years ago
"",
" function peg$buildError() {",
" var expected = peg$expected[0];",
" var failPos = expected.pos;",
"",
" return peg$buildStructuredError(",
" expected.variants,",
" failPos < input.length ? input.charAt(failPos) : null,",
" failPos < input.length",
" ? peg$computeLocation(failPos, failPos + 1)",
" : peg$computeLocation(failPos, failPos)",
" );",
" }",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
""
].join( "\n" ) );
if ( options.optimize === "size" ) {
parts.push( indent2( generateInterpreter() ) );
parts.push( "" );
Code generator rewrite This is a complete rewrite of the PEG.js code generator. Its goals are: 1. Allow optimizing the generated parser code for code size as well as for parsing speed. 2. Prepare ground for future optimizations and big features (like incremental parsing). 2. Replace the old template-based code-generation system with something more lightweight and flexible. 4. General code cleanup (structure, style, variable names, ...). New Architecture ---------------- The new code generator consists of two steps: * Bytecode generator -- produces bytecode for an abstract virtual machine * JavaScript generator -- produces JavaScript code based on the bytecode The abstract virtual machine is stack-based. Originally I wanted to make it register-based, but it turned out that all the code related to it would be more complex and the bytecode itself would be longer (because of explicit register specifications in instructions). The only downsides of the stack-based approach seem to be few small inefficiencies (see e.g. the |NIP| instruction), which seem to be insignificant. The new generator allows optimizing for parsing speed or code size (you can choose using the |optimize| option of the |PEG.buildParser| method or the --optimize/-o option on the command-line). When optimizing for size, the JavaScript generator emits the bytecode together with its constant table and a generic bytecode interpreter. Because the interpreter is small and the bytecode and constant table grow only slowly with size of the grammar, the resulting parser is also small. When optimizing for speed, the JavaScript generator just compiles the bytecode into JavaScript. The generated code is relatively efficient, so the resulting parser is fast. Internal Identifiers -------------------- As a small bonus, all internal identifiers visible to user code in the initializer, actions and predicates are prefixed by |peg$|. This lowers the chance that identifiers in user code will conflict with the ones from PEG.js. It also makes using any internals in user code ugly, which is a good thing. This solves GH-92. Performance ----------- The new code generator improved parsing speed and parser code size significantly. The generated parsers are now: * 39% faster when optimizing for speed * 69% smaller when optimizing for size (without minification) * 31% smaller when optimizing for size (with minification) (Parsing speed was measured using the |benchmark/run| script. Code size was measured by generating parsers for examples in the |examples| directory and adding up the file sizes. Minification was done by |uglify --ascii| in version 1.3.4.) Final Note ---------- This is just a beginning! The new code generator lays a foundation upon which many optimizations and improvements can (and will) be made. Stay tuned :-)
12 years ago
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
} else {
Code generator rewrite This is a complete rewrite of the PEG.js code generator. Its goals are: 1. Allow optimizing the generated parser code for code size as well as for parsing speed. 2. Prepare ground for future optimizations and big features (like incremental parsing). 2. Replace the old template-based code-generation system with something more lightweight and flexible. 4. General code cleanup (structure, style, variable names, ...). New Architecture ---------------- The new code generator consists of two steps: * Bytecode generator -- produces bytecode for an abstract virtual machine * JavaScript generator -- produces JavaScript code based on the bytecode The abstract virtual machine is stack-based. Originally I wanted to make it register-based, but it turned out that all the code related to it would be more complex and the bytecode itself would be longer (because of explicit register specifications in instructions). The only downsides of the stack-based approach seem to be few small inefficiencies (see e.g. the |NIP| instruction), which seem to be insignificant. The new generator allows optimizing for parsing speed or code size (you can choose using the |optimize| option of the |PEG.buildParser| method or the --optimize/-o option on the command-line). When optimizing for size, the JavaScript generator emits the bytecode together with its constant table and a generic bytecode interpreter. Because the interpreter is small and the bytecode and constant table grow only slowly with size of the grammar, the resulting parser is also small. When optimizing for speed, the JavaScript generator just compiles the bytecode into JavaScript. The generated code is relatively efficient, so the resulting parser is fast. Internal Identifiers -------------------- As a small bonus, all internal identifiers visible to user code in the initializer, actions and predicates are prefixed by |peg$|. This lowers the chance that identifiers in user code will conflict with the ones from PEG.js. It also makes using any internals in user code ugly, which is a good thing. This solves GH-92. Performance ----------- The new code generator improved parsing speed and parser code size significantly. The generated parsers are now: * 39% faster when optimizing for speed * 69% smaller when optimizing for size (without minification) * 31% smaller when optimizing for size (with minification) (Parsing speed was measured using the |benchmark/run| script. Code size was measured by generating parsers for examples in the |examples| directory and adding up the file sizes. Minification was done by |uglify --ascii| in version 1.3.4.) Final Note ---------- This is just a beginning! The new code generator lays a foundation upon which many optimizations and improvements can (and will) be made. Stay tuned :-)
12 years ago
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
ast.rules.forEach( rule => {
Improve location info in tracing events Replace |line|, |column|, and |offset| properties of tracing events with the |location| property. It contains an object similar to the one returned by the |location| function available in action code: { start: { offset: 23, line: 5, column: 6 }, end: { offset: 25, line: 5, column: 8 } } For the |rule.match| event, |start| refers to the position at the beginning of the matched input and |end| refers to the position after the end of the matched input. For |rule.enter| and |rule.fail| events, both |start| and |end| refer to the current position at the time the rule was entered.
10 years ago
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
parts.push( indent2( generateRuleFunction( rule ) ) );
parts.push( "" );
Use one var/let/const per variable (fixup) Fix few instances in generated code which I missed in the previous commit. See #443.
8 years ago
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
} );
Don't use "arrays" in lib/compiler/passes/generate-js.js The "arrays" module will be removed. See #442.
8 years ago
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
}
Code generator rewrite This is a complete rewrite of the PEG.js code generator. Its goals are: 1. Allow optimizing the generated parser code for code size as well as for parsing speed. 2. Prepare ground for future optimizations and big features (like incremental parsing). 2. Replace the old template-based code-generation system with something more lightweight and flexible. 4. General code cleanup (structure, style, variable names, ...). New Architecture ---------------- The new code generator consists of two steps: * Bytecode generator -- produces bytecode for an abstract virtual machine * JavaScript generator -- produces JavaScript code based on the bytecode The abstract virtual machine is stack-based. Originally I wanted to make it register-based, but it turned out that all the code related to it would be more complex and the bytecode itself would be longer (because of explicit register specifications in instructions). The only downsides of the stack-based approach seem to be few small inefficiencies (see e.g. the |NIP| instruction), which seem to be insignificant. The new generator allows optimizing for parsing speed or code size (you can choose using the |optimize| option of the |PEG.buildParser| method or the --optimize/-o option on the command-line). When optimizing for size, the JavaScript generator emits the bytecode together with its constant table and a generic bytecode interpreter. Because the interpreter is small and the bytecode and constant table grow only slowly with size of the grammar, the resulting parser is also small. When optimizing for speed, the JavaScript generator just compiles the bytecode into JavaScript. The generated code is relatively efficient, so the resulting parser is fast. Internal Identifiers -------------------- As a small bonus, all internal identifiers visible to user code in the initializer, actions and predicates are prefixed by |peg$|. This lowers the chance that identifiers in user code will conflict with the ones from PEG.js. It also makes using any internals in user code ugly, which is a good thing. This solves GH-92. Performance ----------- The new code generator improved parsing speed and parser code size significantly. The generated parsers are now: * 39% faster when optimizing for speed * 69% smaller when optimizing for size (without minification) * 31% smaller when optimizing for size (with minification) (Parsing speed was measured using the |benchmark/run| script. Code size was measured by generating parsers for examples in the |examples| directory and adding up the file sizes. Minification was done by |uglify --ascii| in version 1.3.4.) Final Note ---------- This is just a beginning! The new code generator lays a foundation upon which many optimizations and improvements can (and will) be made. Stay tuned :-)
12 years ago
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
if ( ast.initializer ) {
Code generator rewrite This is a complete rewrite of the PEG.js code generator. Its goals are: 1. Allow optimizing the generated parser code for code size as well as for parsing speed. 2. Prepare ground for future optimizations and big features (like incremental parsing). 2. Replace the old template-based code-generation system with something more lightweight and flexible. 4. General code cleanup (structure, style, variable names, ...). New Architecture ---------------- The new code generator consists of two steps: * Bytecode generator -- produces bytecode for an abstract virtual machine * JavaScript generator -- produces JavaScript code based on the bytecode The abstract virtual machine is stack-based. Originally I wanted to make it register-based, but it turned out that all the code related to it would be more complex and the bytecode itself would be longer (because of explicit register specifications in instructions). The only downsides of the stack-based approach seem to be few small inefficiencies (see e.g. the |NIP| instruction), which seem to be insignificant. The new generator allows optimizing for parsing speed or code size (you can choose using the |optimize| option of the |PEG.buildParser| method or the --optimize/-o option on the command-line). When optimizing for size, the JavaScript generator emits the bytecode together with its constant table and a generic bytecode interpreter. Because the interpreter is small and the bytecode and constant table grow only slowly with size of the grammar, the resulting parser is also small. When optimizing for speed, the JavaScript generator just compiles the bytecode into JavaScript. The generated code is relatively efficient, so the resulting parser is fast. Internal Identifiers -------------------- As a small bonus, all internal identifiers visible to user code in the initializer, actions and predicates are prefixed by |peg$|. This lowers the chance that identifiers in user code will conflict with the ones from PEG.js. It also makes using any internals in user code ugly, which is a good thing. This solves GH-92. Performance ----------- The new code generator improved parsing speed and parser code size significantly. The generated parsers are now: * 39% faster when optimizing for speed * 69% smaller when optimizing for size (without minification) * 31% smaller when optimizing for size (with minification) (Parsing speed was measured using the |benchmark/run| script. Code size was measured by generating parsers for examples in the |examples| directory and adding up the file sizes. Minification was done by |uglify --ascii| in version 1.3.4.) Final Note ---------- This is just a beginning! The new code generator lays a foundation upon which many optimizations and improvements can (and will) be made. Stay tuned :-)
12 years ago
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
parts.push( indent2( ast.initializer.code ) );
parts.push( "" );
UMD parsers: Generate parser wrapper separately from its toplevel code Code which was at the toplevel of the "generateJS" function in the code generator is now split into "generateToplevel" (which genreates parser toplevel code) and "generateWrapper" (which generates a wrapper around it). This is pure refactoring, generated parser code is exactly the same as before. This change will make it easier to modifiy the code genreator to produce UMD modules. Part of work on #362.
9 years ago
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
}
Implement basic support for tracing Parsers can now be generated with support for tracing using the --trace CLI option or a boolean |trace| option to |PEG.buildParser|. This makes them trace their progress, which can be useful for debugging. Parsers generated with tracing support are called "tracing parsers". When a tracing parser executes, by default it traces the rules it enters and exits by writing messages to the console. For example, a parser built from this grammar: start = a / b a = "a" b = "b" will write this to the console when parsing input "b": 1:1 rule.enter start 1:1 rule.enter a 1:1 rule.fail a 1:1 rule.enter b 1:2 rule.match b 1:2 rule.match start You can customize tracing by passing a custom *tracer* to parser's |parse| method using the |tracer| option: parser.parse(input, { trace: tracer }); This will replace the built-in default tracer (which writes to the console) by the tracer you supplied. The tracer must be an object with a |trace| method. This method is called each time a tracing event happens. It takes one argument which is an object describing the tracing event. Currently, three events are supported: * rule.enter -- triggered when a rule is entered * rule.match -- triggered when a rule matches successfully * rule.fail -- triggered when a rule fails to match These events are triggered in nested pairs -- for each rule.enter event there is a matching rule.match or rule.fail event. The event object passed as an argument to |trace| contains these properties: * type -- event type * rule -- name of the rule the event is related to * offset -- parse position at the time of the event * line -- line at the time of the event * column -- column at the time of the event * result -- rule's match result (only for rule.match event) The whole tracing API is somewhat experimental (which is why it isn't documented properly yet) and I expect it will evolve over time as experience is gained. The default tracer is also somewhat bare-bones. I hope that PEG.js user community will develop more sophisticated tracers over time and I'll be able to integrate their best ideas into the default tracer.
10 years ago
Partial fix #194 - nonsenses error messages with semantic predicates (#547) * Remove stack manipulations from FAIL opcode and rename FAIL to EXPECT * Always save the expected values regardless of the match result of expression * Remove unnecessary check for match result in the `named` node * Collect and process expectations from predicates
7 years ago
parts.push( " peg$begin();" );
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
if ( options.optimize === "size" ) {
Code generator rewrite This is a complete rewrite of the PEG.js code generator. Its goals are: 1. Allow optimizing the generated parser code for code size as well as for parsing speed. 2. Prepare ground for future optimizations and big features (like incremental parsing). 2. Replace the old template-based code-generation system with something more lightweight and flexible. 4. General code cleanup (structure, style, variable names, ...). New Architecture ---------------- The new code generator consists of two steps: * Bytecode generator -- produces bytecode for an abstract virtual machine * JavaScript generator -- produces JavaScript code based on the bytecode The abstract virtual machine is stack-based. Originally I wanted to make it register-based, but it turned out that all the code related to it would be more complex and the bytecode itself would be longer (because of explicit register specifications in instructions). The only downsides of the stack-based approach seem to be few small inefficiencies (see e.g. the |NIP| instruction), which seem to be insignificant. The new generator allows optimizing for parsing speed or code size (you can choose using the |optimize| option of the |PEG.buildParser| method or the --optimize/-o option on the command-line). When optimizing for size, the JavaScript generator emits the bytecode together with its constant table and a generic bytecode interpreter. Because the interpreter is small and the bytecode and constant table grow only slowly with size of the grammar, the resulting parser is also small. When optimizing for speed, the JavaScript generator just compiles the bytecode into JavaScript. The generated code is relatively efficient, so the resulting parser is fast. Internal Identifiers -------------------- As a small bonus, all internal identifiers visible to user code in the initializer, actions and predicates are prefixed by |peg$|. This lowers the chance that identifiers in user code will conflict with the ones from PEG.js. It also makes using any internals in user code ugly, which is a good thing. This solves GH-92. Performance ----------- The new code generator improved parsing speed and parser code size significantly. The generated parsers are now: * 39% faster when optimizing for speed * 69% smaller when optimizing for size (without minification) * 31% smaller when optimizing for size (with minification) (Parsing speed was measured using the |benchmark/run| script. Code size was measured by generating parsers for examples in the |examples| directory and adding up the file sizes. Minification was done by |uglify --ascii| in version 1.3.4.) Final Note ---------- This is just a beginning! The new code generator lays a foundation upon which many optimizations and improvements can (and will) be made. Stay tuned :-)
12 years ago
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
parts.push( " peg$result = peg$parseRule(peg$startRuleIndex);" );
Code generator rewrite This is a complete rewrite of the PEG.js code generator. Its goals are: 1. Allow optimizing the generated parser code for code size as well as for parsing speed. 2. Prepare ground for future optimizations and big features (like incremental parsing). 2. Replace the old template-based code-generation system with something more lightweight and flexible. 4. General code cleanup (structure, style, variable names, ...). New Architecture ---------------- The new code generator consists of two steps: * Bytecode generator -- produces bytecode for an abstract virtual machine * JavaScript generator -- produces JavaScript code based on the bytecode The abstract virtual machine is stack-based. Originally I wanted to make it register-based, but it turned out that all the code related to it would be more complex and the bytecode itself would be longer (because of explicit register specifications in instructions). The only downsides of the stack-based approach seem to be few small inefficiencies (see e.g. the |NIP| instruction), which seem to be insignificant. The new generator allows optimizing for parsing speed or code size (you can choose using the |optimize| option of the |PEG.buildParser| method or the --optimize/-o option on the command-line). When optimizing for size, the JavaScript generator emits the bytecode together with its constant table and a generic bytecode interpreter. Because the interpreter is small and the bytecode and constant table grow only slowly with size of the grammar, the resulting parser is also small. When optimizing for speed, the JavaScript generator just compiles the bytecode into JavaScript. The generated code is relatively efficient, so the resulting parser is fast. Internal Identifiers -------------------- As a small bonus, all internal identifiers visible to user code in the initializer, actions and predicates are prefixed by |peg$|. This lowers the chance that identifiers in user code will conflict with the ones from PEG.js. It also makes using any internals in user code ugly, which is a good thing. This solves GH-92. Performance ----------- The new code generator improved parsing speed and parser code size significantly. The generated parsers are now: * 39% faster when optimizing for speed * 69% smaller when optimizing for size (without minification) * 31% smaller when optimizing for size (with minification) (Parsing speed was measured using the |benchmark/run| script. Code size was measured by generating parsers for examples in the |examples| directory and adding up the file sizes. Minification was done by |uglify --ascii| in version 1.3.4.) Final Note ---------- This is just a beginning! The new code generator lays a foundation upon which many optimizations and improvements can (and will) be made. Stay tuned :-)
12 years ago
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
} else {
UMD parsers: Generate parser wrapper separately from its toplevel code Code which was at the toplevel of the "generateJS" function in the code generator is now split into "generateToplevel" (which genreates parser toplevel code) and "generateWrapper" (which generates a wrapper around it). This is pure refactoring, generated parser code is exactly the same as before. This change will make it easier to modifiy the code genreator to produce UMD modules. Part of work on #362.
9 years ago
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
parts.push( " peg$result = peg$startRuleFunction();" );
Code generator rewrite This is a complete rewrite of the PEG.js code generator. Its goals are: 1. Allow optimizing the generated parser code for code size as well as for parsing speed. 2. Prepare ground for future optimizations and big features (like incremental parsing). 2. Replace the old template-based code-generation system with something more lightweight and flexible. 4. General code cleanup (structure, style, variable names, ...). New Architecture ---------------- The new code generator consists of two steps: * Bytecode generator -- produces bytecode for an abstract virtual machine * JavaScript generator -- produces JavaScript code based on the bytecode The abstract virtual machine is stack-based. Originally I wanted to make it register-based, but it turned out that all the code related to it would be more complex and the bytecode itself would be longer (because of explicit register specifications in instructions). The only downsides of the stack-based approach seem to be few small inefficiencies (see e.g. the |NIP| instruction), which seem to be insignificant. The new generator allows optimizing for parsing speed or code size (you can choose using the |optimize| option of the |PEG.buildParser| method or the --optimize/-o option on the command-line). When optimizing for size, the JavaScript generator emits the bytecode together with its constant table and a generic bytecode interpreter. Because the interpreter is small and the bytecode and constant table grow only slowly with size of the grammar, the resulting parser is also small. When optimizing for speed, the JavaScript generator just compiles the bytecode into JavaScript. The generated code is relatively efficient, so the resulting parser is fast. Internal Identifiers -------------------- As a small bonus, all internal identifiers visible to user code in the initializer, actions and predicates are prefixed by |peg$|. This lowers the chance that identifiers in user code will conflict with the ones from PEG.js. It also makes using any internals in user code ugly, which is a good thing. This solves GH-92. Performance ----------- The new code generator improved parsing speed and parser code size significantly. The generated parsers are now: * 39% faster when optimizing for speed * 69% smaller when optimizing for size (without minification) * 31% smaller when optimizing for size (with minification) (Parsing speed was measured using the |benchmark/run| script. Code size was measured by generating parsers for examples in the |examples| directory and adding up the file sizes. Minification was done by |uglify --ascii| in version 1.3.4.) Final Note ---------- This is just a beginning! The new code generator lays a foundation upon which many optimizations and improvements can (and will) be made. Stay tuned :-)
12 years ago
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
}
UMD parsers: Generate parser wrapper separately from its toplevel code Code which was at the toplevel of the "generateJS" function in the code generator is now split into "generateToplevel" (which genreates parser toplevel code) and "generateWrapper" (which generates a wrapper around it). This is pure refactoring, generated parser code is exactly the same as before. This change will make it easier to modifiy the code genreator to produce UMD modules. Part of work on #362.
9 years ago
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
parts.push( [
"",
" if (peg$result !== peg$FAILED && peg$currPos === input.length) {",
" return peg$result;",
" } else {",
" if (peg$result !== peg$FAILED && peg$currPos < input.length) {",
Partial fix #194 - nonsenses error messages with semantic predicates (#547) * Remove stack manipulations from FAIL opcode and rename FAIL to EXPECT * Always save the expected values regardless of the match result of expression * Remove unnecessary check for match result in the `named` node * Collect and process expectations from predicates
7 years ago
" peg$expect(peg$endExpectation());",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
" }",
"",
Partial fix #194 - nonsenses error messages with semantic predicates (#547) * Remove stack manipulations from FAIL opcode and rename FAIL to EXPECT * Always save the expected values regardless of the match result of expression * Remove unnecessary check for match result in the `named` node * Collect and process expectations from predicates
7 years ago
" throw peg$buildError();",
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
" }",
"}"
].join( "\n" ) );
return parts.join( "\n" );
UMD parsers: Generate parser wrapper separately from its toplevel code Code which was at the toplevel of the "generateJS" function in the code generator is now split into "generateToplevel" (which genreates parser toplevel code) and "generateWrapper" (which generates a wrapper around it). This is pure refactoring, generated parser code is exactly the same as before. This change will make it easier to modifiy the code genreator to produce UMD modules. Part of work on #362.
9 years ago
Add support for ES modules
8 years ago
}
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
function generateWrapper( toplevelCode ) {
Added 'header' option (#491)
7 years ago
function generateHeaderComment() {
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
Set version to v0.11.0-dev
7 years ago
let comment = "// Generated by PEG.js v0.11.0-dev, https://pegjs.org/";
Added 'header' option (#491)
7 years ago
const header = options.header;
if ( typeof header === "string" ) {
comment += "\n\n" + header;
} else if ( Array.isArray( header ) ) {
comment += "\n\n";
header.forEach( data => {
comment += "// " + data;
} );
}
return comment;
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
peg.generate: Implement { format: "node" }
8 years ago
}
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
function generateParserObject() {
return options.trace
? [
"{",
" SyntaxError: peg$SyntaxError,",
" DefaultTracer: peg$DefaultTracer,",
" parse: peg$parse",
"}"
].join( "\n" )
: [
"{",
" SyntaxError: peg$SyntaxError,",
" parse: peg$parse",
"}"
].join( "\n" );
Add support for ES modules
8 years ago
}
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
function generateParserExports() {
return options.trace
? [
"{",
" peg$SyntaxError as SyntaxError,",
" peg$DefaultTracer as DefaultTracer,",
" peg$parse as parse",
"}"
].join( "\n" )
: [
"{",
" peg$SyntaxError as SyntaxError,",
" peg$parse as parse",
"}"
].join( "\n" );
UMD parsers: Allow generating parsers in UMD format from the API Introduce new "format" and "exportVar" options to PEG.buildParser which together allow generating parsers in UMD format. Part of work on #362.
9 years ago
}
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
const generators = {
bare() {
return [
Added 'header' option (#491)
7 years ago
generateHeaderComment(),
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
"(function() {",
" \"use strict\";",
"",
indent2( toplevelCode ),
"",
indent2( "return " + generateParserObject() + ";" ),
"})()"
].join( "\n" );
},
commonjs() {
const parts = [];
const dependencyVars = Object.keys( options.dependencies );
parts.push( [
Added 'header' option (#491)
7 years ago
generateHeaderComment(),
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
"",
"\"use strict\";",
""
].join( "\n" ) );
if ( dependencyVars.length > 0 ) {
dependencyVars.forEach( variable => {
parts.push( "var " + variable
+ " = require(\""
+ js.stringEscape( options.dependencies[ variable ] )
+ "\");"
);
} );
parts.push( "" );
}
parts.push( [
toplevelCode,
"",
"module.exports = " + generateParserObject() + ";",
""
].join( "\n" ) );
return parts.join( "\n" );
},
es() {
const parts = [];
const dependencyVars = Object.keys( options.dependencies );
parts.push(
Added 'header' option (#491)
7 years ago
generateHeaderComment(),
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
""
);
if ( dependencyVars.length > 0 ) {
dependencyVars.forEach( variable => {
parts.push( "import " + variable
+ " from \""
+ js.stringEscape( options.dependencies[ variable ] )
+ "\";"
);
} );
parts.push( "" );
}
parts.push(
toplevelCode,
"",
"export " + generateParserExports() + ";",
"",
"export default " + generateParserObject() + ";",
""
);
return parts.join( "\n" );
},
amd() {
const dependencyVars = Object.keys( options.dependencies );
const dependencyIds = dependencyVars.map( v => options.dependencies[ v ] );
const dependencies = "["
+ dependencyIds
.map( id => `"${ js.stringEscape( id ) }"` )
.join( ", " )
+ "]";
const params = dependencyVars.join( ", " );
return [
Added 'header' option (#491)
7 years ago
generateHeaderComment(),
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
"define(" + dependencies + ", function(" + params + ") {",
" \"use strict\";",
"",
indent2( toplevelCode ),
"",
indent2( "return " + generateParserObject() + ";" ),
"});",
""
].join( "\n" );
},
globals() {
return [
Added 'header' option (#491)
7 years ago
generateHeaderComment(),
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
"(function(root) {",
" \"use strict\";",
"",
indent2( toplevelCode ),
"",
indent2( "root." + options.exportVar + " = " + generateParserObject() + ";" ),
"})(this);",
""
].join( "\n" );
},
umd() {
const parts = [];
const dependencyVars = Object.keys( options.dependencies );
const dependencyIds = dependencyVars.map( v => options.dependencies[ v ] );
const dependencies = "["
+ dependencyIds
.map( id => `"${ js.stringEscape( id ) }"` )
.join( ", " )
+ "]";
const requires = dependencyIds
.map( id => `require("${ js.stringEscape( id ) }")` )
.join( ", " );
Fix shadowing issue on UMD format (#499)
7 years ago
const args = dependencyVars.map( v => "root." + v ).join( ", " );
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
const params = dependencyVars.join( ", " );
parts.push( [
Added 'header' option (#491)
7 years ago
generateHeaderComment(),
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
"(function(root, factory) {",
" if (typeof define === \"function\" && define.amd) {",
" define(" + dependencies + ", factory);",
" } else if (typeof module === \"object\" && module.exports) {",
" module.exports = factory(" + requires + ");"
].join( "\n" ) );
if ( options.exportVar !== null ) {
parts.push( [
" } else {",
Fix shadowing issue on UMD format (#499)
7 years ago
" root." + options.exportVar + " = factory(" + args + ");"
Update code format and style This is related to my last commit. I've updated all the JavaScript files to satisfy 'eslint-config-futagozaryuu', my eslint configuration. I'm sure I've probally missed something, but I've run all NPM scripts and Gulp tasks, fixed any bugs that cropped up, and updated some stuff (mainly related to generated messages), so as far as I can, tell this conversion is over (I know I've probally jixed it just by saying this ;P).
7 years ago
].join( "\n" ) );
}
parts.push( [
" }",
"})(this, function(" + params + ") {",
" \"use strict\";",
"",
indent2( toplevelCode ),
"",
indent2( "return " + generateParserObject() + ";" ),
"});",
""
].join( "\n" ) );
return parts.join( "\n" );
}
};
return generators[ options.format ]();
}
ast.code = generateWrapper( generateToplevel() );
Change module exporting style Modules now generally store the exported object in a named variable or function first and only assign |module.exports| at the very end. This is a difference when compared to style used until now, where most modules started with a |module.exports| assignment. I think the explicit name helps readability and understandability.
11 years ago
}
Rename generate-javascript.js to generate-js.js Short & sweet.
9 years ago
module.exports = generateJS;