Browse Source

Rewrote command line tool

- Split into 3 files: "peg.js", "options.js" and "usage.txt"
- Rewrote arguments parser and helpers to be more precise
- Any arguments after "--" will be passed to "options['--']" now
- Added negation options: "--no-cache" and "--no-trace"
- Added "bare" to accepted module formats
- Added 2 aliases for "--extra-options-file": "-c" and "--config"
- Added short options: "-a", "-f" and "-p"
- Reformatted help text in "usage.txt"
- Updated documentation related to command line options and normal options
- Changed "bin" field in "package.json" from "bin/pegjs" to "bin/peg"
- Added documentation note about command line options that are repeated
- Updated gulpfile.js, replacing "bin/pegjs" with "bin/*.js"

See #429, which was what I intended to fix/solve, but instead pushed back and did this instead.
master
Futago-za Ryuu 4 years ago
parent
commit
0ed8c6f89a
  1. 65
      README.md
  2. 260
      bin/options.js
  3. 64
      bin/peg.js
  4. 329
      bin/pegjs
  5. 51
      bin/usage.txt
  6. 2
      gulpfile.js
  7. 2
      package.json

65
README.md

@ -123,26 +123,24 @@ override this using the `--format` option.
You can tweak the generated parser with several options:
* `--allowed-start-rules` — comma-separated list of rules the parser will be
allowed to start parsing from (default: the first rule in the grammar)
* `--cache` — makes the parser cache results, avoiding exponential parsing
time in pathological cases but making the parser slower
* `--dependency` — makes the parser require a specified dependency (can be
specified multiple times)
* `--export-var` — name of a global variable into which the parser object is
assigned to when no module loader is detected
* `--extra-options` — additional options (in JSON format) to pass to
`peg.generate`
* `--extra-options-file` — file with additional options (in JSON format) to
pass to `peg.generate`
* `--format` — format of the generated parser: `amd`, `commonjs`, `es`,
`globals`, `umd` (default: `commonjs`)
* `--optimize` — selects between optimizing the generated parser for parsing
speed (`speed`) or code size (`size`) (default: `speed`)
* `--plugin` — makes PEG.js use a specified plugin (can be specified multiple
times)
* `-a`, `--allowed-start-rules` — comma-separated list of rules the parser will be allowed to start parsing from (default: the first rule in the grammar)
* `--cache` — makes the parser cache results, avoiding exponential parsing time in pathological cases but making the parser slower
* `-d`, `--dependency` — makes the parser require a specified dependency (can be specified multiple times)
* `-e`, `--export-var` — name of a global variable into which the parser object is assigned to when no module loader is detected
* `--extra-options` — additional options (in JSON format) to pass to `peg.generate`
* `-c`, `--config`, `--extra-options-file` — file with additional options (in JSON format) to pass to `peg.generate`
* `-f`, `--format` — format of the generated parser: `amd`, `bare`, `commonjs`, `es`, `globals`, `umd` (default: `commonjs`)
* `-O`, `--optimize` — selects between optimizing the generated parser for parsing speed (`speed`) or code size (`size`) (default: `speed`)
* `-p`, `--plugin` — makes PEG.js use a specified plugin (can be specified multiple times)
* `--trace` — makes the parser trace its progress
**NOTE:** On the command line, unless it's a repeatable option, any option on the right side will take priority over either the same option mentioned
before or it's counter part:
- `pegjs -f es -f bare` will set `options.format` to `bare`
- `pegjs --no-trace --trace` will set `options.trace` to `true`
- `pegjs -a start,Rule -a Rule,Template` will set `options.allowedStartRules` to `[ "start", "Rule", "Template" ]`
### JavaScript API
In Node.js, require the PEG.js parser generator module:
@ -170,26 +168,17 @@ property with more details about the error.
You can tweak the generated parser by passing a second parameter with an options
object to `peg.generate`. The following options are supported:
* `allowedStartRules` — rules the parser will be allowed to start parsing from
(default: the first rule in the grammar)
* `cache` — if `true`, makes the parser cache results, avoiding exponential
parsing time in pathological cases but making the parser slower (default:
`false`)
* `dependencies` — parser dependencies, the value is an object which maps
variables used to access the dependencies in the parser to module IDs used
to load them; valid only when `format` is set to `"amd"`, `"commonjs"`,
`"es"`, or `"umd"` (default: `{}`)
* `exportVar` — name of a global variable into which the parser object is
assigned to when no module loader is detected; valid only when `format` is
set to `"globals"` or `"umd"` (default: `null`)
* `format` — format of the generated parser (`"amd"`, `"bare"`, `"commonjs"`,
`"es"`, `"globals"`, or `"umd"`); valid only when `output` is set to
`"source"` (default: `"bare"`)
* `optimize`— selects between optimizing the generated parser for parsing
speed (`"speed"`) or code size (`"size"`) (default: `"speed"`)
* `output` — if set to `"parser"`, the method will return generated parser
object; if set to `"source"`, it will return parser source code as a string
(default: `"parser"`)
* `allowedStartRules` — rules the parser will be allowed to start parsing from (default: the first rule in the grammar)
* `cache` — if `true`, makes the parser cache results, avoiding exponential parsing time in pathological cases but making the parser slower (default: `false`)
* `dependencies` — parser dependencies, the value is an object which maps variables used to access the dependencies to module IDs used to load them;<br>
valid only when `format` is set to `"amd"`, `"commonjs"`, `"es"`, or `"umd"` (default: `{}`)
* `exportVar` — name of an optional global variable into which the parser object is assigned to when no module loader is detected;
valid only when `format` is set to `"globals"` or `"umd"`
* `format` — format of the generated parser (`"amd"`, `"bare"`, `"commonjs"`, `"es"`, `"globals"`, or `"umd"`);
valid only when `output` is set to `"source"` (default: `"bare"`)
* `optimize`— selects between optimizing the generated parser for parsing speed (`"speed"`) or code size (`"size"`) (default: `"speed"`)
* `output` — if set to `"parser"` (default), the method will return generated parser object;
if set to `"source"`, it will return parser source code as a string
* `plugins` — plugins to use
* `trace` — makes the parser trace its progress (default: `false`)

260
bin/options.js

@ -0,0 +1,260 @@
"use strict";
let fs = require("fs");
let path = require("path");
let peg = require("../");
// Options
let inputFile = null;
let outputFile = null;
let options = {
"--": [],
"cache": false,
"dependencies": {},
"exportVar": null,
"format": "commonjs",
"optimize": "speed",
"output": "source",
"plugins": [],
"trace": false
};
const EXPORT_VAR_FORMATS = ["globals", "umd"];
const DEPENDENCY_FORMATS = ["amd", "commonjs", "es", "umd"];
const MODULE_FORMATS = ["amd", "bare", "commonjs", "es", "globals", "umd"];
const OPTIMIZATION_GOALS = ["size", "speed"];
// Helpers
function abort(message) {
console.error(message);
process.exit(1);
}
function addExtraOptions(json) {
let extraOptions;
try {
extraOptions = JSON.parse(json);
} catch (e) {
if (!(e instanceof SyntaxError)) { throw e; }
abort("Error parsing JSON: " + e.message);
}
if (typeof extraOptions !== "object") {
abort("The JSON with extra options has to represent an object.");
}
Object
.keys(extraOptions)
.forEach(key => {
options[key] = extraOptions[key];
});
}
function formatChoicesList(list) {
list = list.map(entry => `"${entry}"`);
let lastOption = list.pop();
return list.length === 0
? lastOption
: list.join(", ") + " or " + lastOption;
}
function updateList(list, string) {
string
.split(",")
.forEach(entry => {
entry = entry.trim();
if (list.indexOf(entry) === -1) {
list.push(entry);
}
});
}
// Arguments
let args = process.argv.slice(2);
function nextArg(option) {
if (args.length === 0) {
abort(`Missing parameter of the ${option} option.`);
}
return args.shift();
}
// Parse Arguments
while (args.length > 0) {
let json, mod;
let argument = args.shift();
if (argument.indexOf("-") === 0 && argument.indexOf("=") > 1) {
argument = argument.split("=");
args.unshift(argument.length > 2 ? argument.slice(1) : argument[1]);
argument = argument[0];
}
switch (argument) {
case "--":
options["--"] = args;
args = [];
break;
case "-a":
case "--allowed-start-rules":
if (!options.allowedStartRules) {
options.allowedStartRules = [];
}
updateList(options.allowedStartRules, nextArg("--allowed-start-rules"));
break;
case "--cache":
options.cache = true;
break;
case "--no-cache":
options.cache = false;
break;
case "-d":
case "--dependency":
argument = nextArg("-d/--dependency");
if (argument.indexOf(":") === -1) {
mod = [argument, argument];
} else {
mod = argument.split(":");
if (mod.length > 2) {
mod[1] = mod.slice(1);
}
}
options.dependencies[mod[0]] = mod[1];
break;
case "-e":
case "--export-var":
options.exportVar = nextArg("-e/--export-var");
break;
case "--extra-options":
addExtraOptions(nextArg("--extra-options"));
break;
case "-c":
case "--config":
case "--extra-options-file":
argument = nextArg("-c/--config/--extra-options-file");
try {
json = fs.readFileSync(argument, "utf8");
} catch (e) {
abort(`Can't read from file "${argument}".`);
}
addExtraOptions(json);
break;
case "-f":
case "--format":
argument = nextArg("-f/--format");
if (MODULE_FORMATS.indexOf(argument) === -1) {
abort(`Module format must be either ${formatChoicesList(MODULE_FORMATS)}.`);
}
options.format = argument;
break;
case "-h":
case "--help":
console.log(fs.readFileSync(path.join(__dirname, "usage.txt"), "utf8").trim());
process.exit();
break;
case "-O":
case "--optimize":
argument = nextArg("-O/--optimize");
if (OPTIMIZATION_GOALS.indexOf(argument) === -1) {
abort(`Optimization goal must be either ${formatChoicesList(OPTIMIZATION_GOALS)}.`);
}
options.optimize = argument;
break;
case "-o":
case "--output":
outputFile = nextArg("-o/--output");
break;
case "-p":
case "--plugin":
argument = nextArg("-p/--plugin");
try {
mod = require(argument);
} catch (ex1) {
if (ex1.code !== "MODULE_NOT_FOUND") { throw ex1; }
try {
mod = require(path.resolve(argument));
} catch (ex2) {
if (ex2.code !== "MODULE_NOT_FOUND") { throw ex2; }
abort(`Can't load module "${argument}".`);
}
}
options.plugins.push(mod);
break;
case "--trace":
options.trace = true;
break;
case "--no-trace":
options.trace = false;
break;
case "-v":
case "--version":
console.log("PEG.js v" + peg.VERSION);
process.exit();
break;
default:
if (inputFile !== null) {
abort(`Unknown option: "${argument}".`);
}
inputFile = argument;
}
}
// Validation and defaults
if (Object.keys(options.dependencies).length > 0) {
if (DEPENDENCY_FORMATS.indexOf(options.format) === -1) {
abort(`Can't use the -d/--dependency option with the "${options.format}" module format.`);
}
}
if (options.exportVar !== null) {
if (EXPORT_VAR_FORMATS.indexOf(options.format) === -1) {
abort(`Can't use the -e/--export-var option with the "${options.format}" module format.`);
}
}
if (inputFile === null) {
inputFile = "-";
}
if (outputFile === null) {
if (inputFile === "-") {
outputFile = "-";
} else if (inputFile) {
outputFile = inputFile.substr(0, inputFile.length - path.extname(inputFile).length) + ".js";
}
}
// Export
options.inputFile = inputFile;
options.outputFile = outputFile;
module.exports = options;

64
bin/peg.js

@ -0,0 +1,64 @@
#!/usr/bin/env node
"use strict";
let fs = require("fs");
let peg = require("../lib/peg");
let options = require("./options");
// Helpers
function readStream(inputStream, callback) {
let input = "";
inputStream.on("data", data => { input += data; });
inputStream.on("end", () => { callback(input); });
}
function abort(message) {
console.error(message);
process.exit(1);
}
// Main
let inputStream, outputStream;
if (options.inputFile === "-") {
process.stdin.resume();
inputStream = process.stdin;
inputStream.on("error", () => {
abort(`Can't read from file "${options.inputFile}".`);
});
} else {
inputStream = fs.createReadStream(options.inputFile);
}
if (options.outputFile === "-") {
outputStream = process.stdout;
} else {
outputStream = fs.createWriteStream(options.outputFile);
outputStream.on("error", () => {
abort(`Can't write to file "${options.outputFile}".`);
});
}
readStream(inputStream, input => {
let location, source;
try {
source = peg.generate(input, options);
} catch (e) {
if (e.location !== undefined) {
location = e.location.start;
abort(location.line + ":" + location.column + ": " + e.message);
} else {
abort(e.message);
}
}
outputStream.write(source);
if (outputStream !== process.stdout) {
outputStream.end();
}
});

329
bin/pegjs

@ -1,329 +0,0 @@
#!/usr/bin/env node
"use strict";
let fs = require("fs");
let path = require("path");
let peg = require("../lib/peg");
// Helpers
function printVersion() {
console.log("PEG.js " + peg.VERSION);
}
function printHelp() {
console.log("Usage: pegjs [options] [--] [<input_file>]");
console.log("");
console.log("Options:");
console.log(" --allowed-start-rules <rules> comma-separated list of rules the generated");
console.log(" parser will be allowed to start parsing");
console.log(" from (default: the first rule in the");
console.log(" grammar)");
console.log(" --cache make generated parser cache results");
console.log(" -d, --dependency <dependency> use specified dependency (can be specified");
console.log(" multiple times)");
console.log(" -e, --export-var <variable> name of a global variable into which the");
console.log(" parser object is assigned to when no module");
console.log(" loader is detected");
console.log(" --extra-options <options> additional options (in JSON format) to pass");
console.log(" to peg.generate");
console.log(" --extra-options-file <file> file with additional options (in JSON");
console.log(" format) to pass to peg.generate");
console.log(" --format <format> format of the generated parser: amd, commonjs");
console.log(" es, globals, umd (default: commonjs)");
console.log(" -h, --help print help and exit");
console.log(" -O, --optimize <goal> select optimization for speed or size");
console.log(" (default: speed)");
console.log(" -o, --output <file> output file");
console.log(" --plugin <plugin> use a specified plugin (can be specified");
console.log(" multiple times)");
console.log(" --trace enable tracing in generated parser");
console.log(" -v, --version print version information and exit");
}
function exitSuccess() {
process.exit(0);
}
function exitFailure() {
process.exit(1);
}
function abort(message) {
console.error(message);
exitFailure();
}
function addExtraOptions(options, json) {
let extraOptions;
try {
extraOptions = JSON.parse(json);
} catch (e) {
if (!(e instanceof SyntaxError)) { throw e; }
abort("Error parsing JSON: " + e.message);
}
if (typeof extraOptions !== "object") {
abort("The JSON with extra options has to represent an object.");
}
Object.keys(extraOptions).forEach(key => {
options[key] = extraOptions[key];
});
}
// Extracted into a function just to silence JSHint complaining about creating
// functions in a loop.
function trim(s) {
return s.trim();
}
// Arguments
let args = process.argv.slice(2); // Trim "node" and the script path.
function isOption(arg) {
return (/^-.+/).test(arg);
}
function nextArg() {
args.shift();
}
// Files
function readStream(inputStream, callback) {
let input = "";
inputStream.on("data", data => { input += data; });
inputStream.on("end", () => { callback(input); });
}
// Main
let inputFile = null;
let outputFile = null;
let options = {
cache: false,
dependencies: {},
exportVar: null,
format: "commonjs",
optimize: "speed",
output: "source",
plugins: [],
trace: false
};
const MODULE_FORMATS = ["amd", "commonjs", "es", "globals", "umd"];
const MODULE_FORMATS_WITH_DEPS = ["amd", "commonjs", "es", "umd"];
while (args.length > 0 && isOption(args[0])) {
let json, id, mod;
switch (args[0]) {
case "--allowed-start-rules":
nextArg();
if (args.length === 0) {
abort("Missing parameter of the -e/--allowed-start-rules option.");
}
options.allowedStartRules = args[0]
.split(",")
.map(trim);
break;
case "--cache":
options.cache = true;
break;
case "-d":
case "--dependency":
nextArg();
if (args.length === 0) {
abort("Missing parameter of the -d/--dependency option.");
}
if (args[0].indexOf(":") !== -1) {
let parts = args[0].split(":");
options.dependencies[parts[0]] = parts[1];
} else {
options.dependencies[args[0]] = args[0];
}
break;
case "-e":
case "--export-var":
nextArg();
if (args.length === 0) {
abort("Missing parameter of the -e/--export-var option.");
}
options.exportVar = args[0];
break;
case "--extra-options":
nextArg();
if (args.length === 0) {
abort("Missing parameter of the --extra-options option.");
}
addExtraOptions(options, args[0]);
break;
case "--extra-options-file":
nextArg();
if (args.length === 0) {
abort("Missing parameter of the --extra-options-file option.");
}
try {
json = fs.readFileSync(args[0]);
} catch (e) {
abort("Can't read from file \"" + args[0] + "\".");
}
addExtraOptions(options, json);
break;
case "--format":
nextArg();
if (args.length === 0) {
abort("Missing parameter of the --format option.");
}
if (MODULE_FORMATS.indexOf(args[0]) === -1) {
abort("Module format must be one of " + MODULE_FORMATS.map(format => `"${format}"`).join(", ") + ".");
}
options.format = args[0];
break;
case "-h":
case "--help":
printHelp();
exitSuccess();
break;
case "-O":
case "--optimize":
nextArg();
if (args.length === 0) {
abort("Missing parameter of the -O/--optimize option.");
}
if (args[0] !== "speed" && args[0] !== "size") {
abort("Optimization goal must be either \"speed\" or \"size\".");
}
options.optimize = args[0];
break;
case "-o":
case "--output":
nextArg();
if (args.length === 0) {
abort("Missing parameter of the -o/--output option.");
}
outputFile = args[0];
break;
case "--plugin":
nextArg();
if (args.length === 0) {
abort("Missing parameter of the --plugin option.");
}
id = /^(\.\/|\.\.\/)/.test(args[0]) ? path.resolve(args[0]) : args[0];
try {
mod = require(id);
} catch (e) {
if (e.code !== "MODULE_NOT_FOUND") { throw e; }
abort("Can't load module \"" + id + "\".");
}
options.plugins.push(mod);
break;
case "--trace":
options.trace = true;
break;
case "-v":
case "--version":
printVersion();
exitSuccess();
break;
case "--":
nextArg();
break;
default:
abort("Unknown option: " + args[0] + ".");
}
nextArg();
}
if (Object.keys(options.dependencies).length > 0) {
if (MODULE_FORMATS_WITH_DEPS.indexOf(options.format) === -1) {
abort("Can't use the -d/--dependency option with the \"" + options.format + "\" module format.");
}
}
if (options.exportVar !== null) {
if (options.format !== "globals" && options.format !== "umd") {
abort("Can't use the -e/--export-var option with the \"" + options.format + "\" module format.");
}
}
let inputStream, outputStream;
switch (args.length) {
case 0:
inputFile = "-";
break;
case 1:
inputFile = args[0];
break;
default:
abort("Too many arguments.");
}
if (outputFile === null) {
if (inputFile === "-") {
outputFile = "-";
} else {
outputFile = inputFile.substr(0, inputFile.length - path.extname(inputFile).length) + ".js";
}
}
if (inputFile === "-") {
process.stdin.resume();
inputStream = process.stdin;
inputStream.on("error", () => {
abort("Can't read from file \"" + inputFile + "\".");
});
} else {
inputStream = fs.createReadStream(inputFile);
}
if (outputFile === "-") {
outputStream = process.stdout;
} else {
outputStream = fs.createWriteStream(outputFile);
outputStream.on("error", () => {
abort("Can't write to file \"" + outputFile + "\".");
});
}
readStream(inputStream, input => {
let source;
try {
source = peg.generate(input, options);
} catch (e) {
if (e.location !== undefined) {
abort(e.location.start.line + ":" + e.location.start.column + ": " + e.message);
} else {
abort(e.message);
}
}
outputStream.write(source);
if (outputStream !== process.stdout) {
outputStream.end();
}
});

51
bin/usage.txt

@ -0,0 +1,51 @@
Usage: pegjs [options] [<input_file>] [--]
Options:
-a, --allowed-start-rules <rules> Comma-separated list of rules the generated
parser will be allowed to start parsing from
(default: first rule in the grammar)
(note: repeatable option)
--cache Make generated parser cache results
-c, --config <file> Aliases for "--extra-options-file"
-d, --dependency <dependency> Use specified dependency
(note: repeatable option)
-e, --export-var <variable> Name of a global variable into which the
parser object is assigned to when no
module loader is detected
--extra-options <options> A string with additional options (in JSON
format) to pass to peg.generate
(note: repeatable option)
--extra-options-file <file> File with additional options (in JSON
format) to pass to peg.generate
(note: repeatable option)
-f, --format <format> Format of the generated parser:
amd, bare, commonjs, es, globals, umd
(default: commonjs)
-h, --help Print help and exit
--no-cache Generated parser doesn't cache results
(default behavior)
--no-trace Disable tracing in generated parser
(default behavior)
-O, --optimize <goal> Select optimization for speed or size
(default: speed)
-o, --output <file> Output file
-p, --plugin <plugin> Use a specified plugin
(note: repeatable option)
--trace Enable tracing in generated parser
-v, --version Print version information and exit

2
gulpfile.js

@ -39,7 +39,7 @@ const JS_FILES = [
"benchmark/run",
"benchmark/server",
"!benchmark/vendor/**/*",
"bin/pegjs",
"bin/*.js",
"gulpfile.js"
];

2
package.json

@ -41,7 +41,7 @@
"package.json"
],
"main": "lib/peg",
"bin": "bin/pegjs",
"bin": "bin/peg",
"repository": "pegjs/pegjs",
"scripts": {
"lint": "gulp lint",

Loading…
Cancel
Save