joepie91
/
pegjs
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
redux
master
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '95ce20ed92'
${ noResults }
pegjs/Makefile

144 lines
6.8 KiB
Makefile
Raw Normal View History Unescape Escape

Makefile: Small reordering Define |PEGJS_VERSION| before it is used. While defining it after its first use was OK technically, it made the code a tiny bit harder to read.
12 years ago
# ===== Variables =====
PEGJS_VERSION = `cat $(VERSION_FILE)`
Git repo npmization: Compose PEG.js from Node.js modules PEG.js source code becomes a set of Node.js modules that include each other as needed. The distribution version is built by bundling these modules together, wrapping them inside a bit of boilerplate code that makes |module.exports| and |require| work. Part of a fix for GH-32.
12 years ago
# ===== Modules =====
# Order matters -- dependencies must be listed before modules dependent on them.
Utility functions cleanup: Split lib/utils.js Split lib/utils.js into multiple files. Some of the functions were generic, these were moved into files in lib/utils. Other funtions were specific for the compiler, these were moved to files in lib/compiler. This commit only moves functions around -- there is no renaming and cleanup performed. Both will come later.
11 years ago
MODULES = utils/arrays \
utils/objects \
utils/classes \
Git repo npmization: Compose PEG.js from Node.js modules PEG.js source code becomes a set of Node.js modules that include each other as needed. The distribution version is built by bundling these modules together, wrapping them inside a bit of boilerplate code that makes |module.exports| and |require| work. Part of a fix for GH-32.
12 years ago
grammar-error \
parser \
Utility functions cleanup: Split lib/utils.js Split lib/utils.js into multiple files. Some of the functions were generic, these were moved into files in lib/utils. Other funtions were specific for the compiler, these were moved to files in lib/compiler. This commit only moves functions around -- there is no renaming and cleanup performed. Both will come later.
11 years ago
compiler/visitor \
Extract the |matchesEmpty| visitor from the |reportLeftRecursion| pass Beside the recursion detector, the visitor will also be used by infinite loop detector. Note the newly created |asts.matchesEmpty| function re-creates the visitor each time it is called, which makes it slower than necessary. This could have been worked around in various ways but I chose to defer that optimization because real-world performance impact is small.
10 years ago
compiler/asts \
Code generator rewrite This is a complete rewrite of the PEG.js code generator. Its goals are: 1. Allow optimizing the generated parser code for code size as well as for parsing speed. 2. Prepare ground for future optimizations and big features (like incremental parsing). 2. Replace the old template-based code-generation system with something more lightweight and flexible. 4. General code cleanup (structure, style, variable names, ...). New Architecture ---------------- The new code generator consists of two steps: * Bytecode generator -- produces bytecode for an abstract virtual machine * JavaScript generator -- produces JavaScript code based on the bytecode The abstract virtual machine is stack-based. Originally I wanted to make it register-based, but it turned out that all the code related to it would be more complex and the bytecode itself would be longer (because of explicit register specifications in instructions). The only downsides of the stack-based approach seem to be few small inefficiencies (see e.g. the |NIP| instruction), which seem to be insignificant. The new generator allows optimizing for parsing speed or code size (you can choose using the |optimize| option of the |PEG.buildParser| method or the --optimize/-o option on the command-line). When optimizing for size, the JavaScript generator emits the bytecode together with its constant table and a generic bytecode interpreter. Because the interpreter is small and the bytecode and constant table grow only slowly with size of the grammar, the resulting parser is also small. When optimizing for speed, the JavaScript generator just compiles the bytecode into JavaScript. The generated code is relatively efficient, so the resulting parser is fast. Internal Identifiers -------------------- As a small bonus, all internal identifiers visible to user code in the initializer, actions and predicates are prefixed by |peg$|. This lowers the chance that identifiers in user code will conflict with the ones from PEG.js. It also makes using any internals in user code ugly, which is a good thing. This solves GH-92. Performance ----------- The new code generator improved parsing speed and parser code size significantly. The generated parsers are now: * 39% faster when optimizing for speed * 69% smaller when optimizing for size (without minification) * 31% smaller when optimizing for size (with minification) (Parsing speed was measured using the |benchmark/run| script. Code size was measured by generating parsers for examples in the |examples| directory and adding up the file sizes. Minification was done by |uglify --ascii| in version 1.3.4.) Final Note ---------- This is just a beginning! The new code generator lays a foundation upon which many optimizations and improvements can (and will) be made. Stay tuned :-)
12 years ago
compiler/opcodes \
Utility functions cleanup: Split lib/utils.js Split lib/utils.js into multiple files. Some of the functions were generic, these were moved into files in lib/utils. Other funtions were specific for the compiler, these were moved to files in lib/compiler. This commit only moves functions around -- there is no renaming and cleanup performed. Both will come later.
11 years ago
compiler/javascript \
Code generator rewrite This is a complete rewrite of the PEG.js code generator. Its goals are: 1. Allow optimizing the generated parser code for code size as well as for parsing speed. 2. Prepare ground for future optimizations and big features (like incremental parsing). 2. Replace the old template-based code-generation system with something more lightweight and flexible. 4. General code cleanup (structure, style, variable names, ...). New Architecture ---------------- The new code generator consists of two steps: * Bytecode generator -- produces bytecode for an abstract virtual machine * JavaScript generator -- produces JavaScript code based on the bytecode The abstract virtual machine is stack-based. Originally I wanted to make it register-based, but it turned out that all the code related to it would be more complex and the bytecode itself would be longer (because of explicit register specifications in instructions). The only downsides of the stack-based approach seem to be few small inefficiencies (see e.g. the |NIP| instruction), which seem to be insignificant. The new generator allows optimizing for parsing speed or code size (you can choose using the |optimize| option of the |PEG.buildParser| method or the --optimize/-o option on the command-line). When optimizing for size, the JavaScript generator emits the bytecode together with its constant table and a generic bytecode interpreter. Because the interpreter is small and the bytecode and constant table grow only slowly with size of the grammar, the resulting parser is also small. When optimizing for speed, the JavaScript generator just compiles the bytecode into JavaScript. The generated code is relatively efficient, so the resulting parser is fast. Internal Identifiers -------------------- As a small bonus, all internal identifiers visible to user code in the initializer, actions and predicates are prefixed by |peg$|. This lowers the chance that identifiers in user code will conflict with the ones from PEG.js. It also makes using any internals in user code ugly, which is a good thing. This solves GH-92. Performance ----------- The new code generator improved parsing speed and parser code size significantly. The generated parsers are now: * 39% faster when optimizing for speed * 69% smaller when optimizing for size (without minification) * 31% smaller when optimizing for size (with minification) (Parsing speed was measured using the |benchmark/run| script. Code size was measured by generating parsers for examples in the |examples| directory and adding up the file sizes. Minification was done by |uglify --ascii| in version 1.3.4.) Final Note ---------- This is just a beginning! The new code generator lays a foundation upon which many optimizations and improvements can (and will) be made. Stay tuned :-)
12 years ago
compiler/passes/generate-bytecode \
compiler/passes/generate-javascript \
Git repo npmization: Compose PEG.js from Node.js modules PEG.js source code becomes a set of Node.js modules that include each other as needed. The distribution version is built by bundling these modules together, wrapping them inside a bit of boilerplate code that makes |module.exports| and |require| work. Part of a fix for GH-32.
12 years ago
compiler/passes/remove-proxy-rules \
compiler/passes/report-left-recursion \
compiler/passes/report-missing-rules \
compiler \
peg
Replace Jakefile with Makefile Doing scripting tasks in JavaScript is painful.
13 years ago
# ===== Directories =====
Assume development dependencies are installed locally This is compatible with what "npm install" does and allows for isolated development environment.
12 years ago
SRC_DIR = src
LIB_DIR = lib
BIN_DIR = bin
BROWSER_DIR = browser
SPEC_DIR = spec
BENCHMARK_DIR = benchmark
NODE_MODULES_DIR = node_modules
NODE_MODULES_BIN_DIR = $(NODE_MODULES_DIR)/.bin
Replace Jakefile with Makefile Doing scripting tasks in JavaScript is painful.
13 years ago
# ===== Files =====
PARSER_SRC_FILE = $(SRC_DIR)/parser.pegjs
Git repo npmization: Make the repo a npm package Includes: * Moving the source code from /src to /lib. * Adding an explicit file list to package.json * Updating the Makefile. * Updating the spec and benchmark suites and their READMEs. Part of a fix for GH-32.
12 years ago
PARSER_OUT_FILE = $(LIB_DIR)/parser.js
Replace Jakefile with Makefile Doing scripting tasks in JavaScript is painful.
13 years ago
Git repo npmization: Make the repo a npm package Includes: * Moving the source code from /src to /lib. * Adding an explicit file list to package.json * Updating the Makefile. * Updating the spec and benchmark suites and their READMEs. Part of a fix for GH-32.
12 years ago
BROWSER_FILE_DEV = $(BROWSER_DIR)/peg-$(PEGJS_VERSION).js
BROWSER_FILE_MIN = $(BROWSER_DIR)/peg-$(PEGJS_VERSION).min.js
Replace Jakefile with Makefile Doing scripting tasks in JavaScript is painful.
13 years ago
Git repo npmization: Make the repo a npm package Includes: * Moving the source code from /src to /lib. * Adding an explicit file list to package.json * Updating the Makefile. * Updating the spec and benchmark suites and their READMEs. Part of a fix for GH-32.
12 years ago
VERSION_FILE = VERSION
Replace Jakefile with Makefile Doing scripting tasks in JavaScript is painful.
13 years ago
# ===== Executables =====
Assume development dependencies are installed locally This is compatible with what "npm install" does and allows for isolated development environment.
12 years ago
JSHINT = $(NODE_MODULES_BIN_DIR)/jshint
UGLIFYJS = $(NODE_MODULES_BIN_DIR)/uglifyjs
JASMINE_NODE = $(NODE_MODULES_BIN_DIR)/jasmine-node
Replace Jakefile with Makefile Doing scripting tasks in JavaScript is painful.
13 years ago
PEGJS = $(BIN_DIR)/pegjs
BENCHMARK_RUN = $(BENCHMARK_DIR)/run
# ===== Targets =====
Git repo npmization: Make the repo a npm package Includes: * Moving the source code from /src to /lib. * Adding an explicit file list to package.json * Updating the Makefile. * Updating the spec and benchmark suites and their READMEs. Part of a fix for GH-32.
12 years ago
# Default target
all: browser
Replace Jakefile with Makefile Doing scripting tasks in JavaScript is painful.
13 years ago
# Generate the grammar parser
parser:
Git repo npmization: Compose PEG.js from Node.js modules PEG.js source code becomes a set of Node.js modules that include each other as needed. The distribution version is built by bundling these modules together, wrapping them inside a bit of boilerplate code that makes |module.exports| and |require| work. Part of a fix for GH-32.
12 years ago
$(PEGJS) $(PARSER_SRC_FILE) $(PARSER_OUT_FILE)
Replace Jakefile with Makefile Doing scripting tasks in JavaScript is painful.
13 years ago
Git repo npmization: Make the repo a npm package Includes: * Moving the source code from /src to /lib. * Adding an explicit file list to package.json * Updating the Makefile. * Updating the spec and benchmark suites and their READMEs. Part of a fix for GH-32.
12 years ago
# Build the browser version of the library
browser:
mkdir -p $(BROWSER_DIR)
rm -f $(BROWSER_FILE_DEV)
rm -f $(BROWSER_FILE_MIN)
Git repo npmization: Compose PEG.js from Node.js modules PEG.js source code becomes a set of Node.js modules that include each other as needed. The distribution version is built by bundling these modules together, wrapping them inside a bit of boilerplate code that makes |module.exports| and |require| work. Part of a fix for GH-32.
12 years ago
# The following code is inspired by CoffeeScript's Cakefile.
Git repo npmization: Make the repo a npm package Includes: * Moving the source code from /src to /lib. * Adding an explicit file list to package.json * Updating the Makefile. * Updating the spec and benchmark suites and their READMEs. Part of a fix for GH-32.
12 years ago
echo '/*' >> $(BROWSER_FILE_DEV)
echo " * PEG.js $(PEGJS_VERSION)" >> $(BROWSER_FILE_DEV)
echo ' *' >> $(BROWSER_FILE_DEV)
Update website URL PEG.js website was moved from http://pegjs.majda.cz/ to http://pegjs.org/.
10 years ago
echo ' * http://pegjs.org/' >> $(BROWSER_FILE_DEV)
Git repo npmization: Make the repo a npm package Includes: * Moving the source code from /src to /lib. * Adding an explicit file list to package.json * Updating the Makefile. * Updating the spec and benchmark suites and their READMEs. Part of a fix for GH-32.
12 years ago
echo ' *' >> $(BROWSER_FILE_DEV)
Update copyright years to 2010-2013
11 years ago
echo ' * Copyright (c) 2010-2013 David Majda' >> $(BROWSER_FILE_DEV)
Makefile: Add missing period at the ned of a sentence
11 years ago
echo ' * Licensed under the MIT license.' >> $(BROWSER_FILE_DEV)
Git repo npmization: Make the repo a npm package Includes: * Moving the source code from /src to /lib. * Adding an explicit file list to package.json * Updating the Makefile. * Updating the spec and benchmark suites and their READMEs. Part of a fix for GH-32.
12 years ago
echo ' */' >> $(BROWSER_FILE_DEV)
echo 'var PEG = (function(undefined) {' >> $(BROWSER_FILE_DEV)
echo ' var modules = {' >> $(BROWSER_FILE_DEV)
echo ' define: function(name, factory) {' >> $(BROWSER_FILE_DEV)
echo ' var dir = name.replace(/(^|\/)[^/]+$$/, "$$1"),' >> $(BROWSER_FILE_DEV)
echo ' module = { exports: {} };' >> $(BROWSER_FILE_DEV)
echo '' >> $(BROWSER_FILE_DEV)
echo ' function require(path) {' >> $(BROWSER_FILE_DEV)
echo ' var name = dir + path,' >> $(BROWSER_FILE_DEV)
echo ' regexp = /[^\/]+\/\.\.\/|\.\//;' >> $(BROWSER_FILE_DEV)
echo '' >> $(BROWSER_FILE_DEV)
echo " /* Can't use /.../g because we can move backwards in the string. */" >> $(BROWSER_FILE_DEV)
echo ' while (regexp.test(name)) {' >> $(BROWSER_FILE_DEV)
echo ' name = name.replace(regexp, "");' >> $(BROWSER_FILE_DEV)
echo ' }' >> $(BROWSER_FILE_DEV)
echo '' >> $(BROWSER_FILE_DEV)
echo ' return modules[name];' >> $(BROWSER_FILE_DEV)
echo ' }' >> $(BROWSER_FILE_DEV)
echo '' >> $(BROWSER_FILE_DEV)
echo ' factory(module, require);' >> $(BROWSER_FILE_DEV)
echo ' this[name] = module.exports;' >> $(BROWSER_FILE_DEV)
echo ' }' >> $(BROWSER_FILE_DEV)
echo ' };' >> $(BROWSER_FILE_DEV)
echo '' >> $(BROWSER_FILE_DEV)
for module in $(MODULES); do \
echo " modules.define(\"$$module\", function(module, require) {" >> $(BROWSER_FILE_DEV); \
sed -e 's/^/ /' lib/$$module.js >> $(BROWSER_FILE_DEV); \
echo ' });' >> $(BROWSER_FILE_DEV); \
echo '' >> $(BROWSER_FILE_DEV); \
Git repo npmization: Compose PEG.js from Node.js modules PEG.js source code becomes a set of Node.js modules that include each other as needed. The distribution version is built by bundling these modules together, wrapping them inside a bit of boilerplate code that makes |module.exports| and |require| work. Part of a fix for GH-32.
12 years ago
done
Git repo npmization: Make the repo a npm package Includes: * Moving the source code from /src to /lib. * Adding an explicit file list to package.json * Updating the Makefile. * Updating the spec and benchmark suites and their READMEs. Part of a fix for GH-32.
12 years ago
echo ' return modules["peg"]' >> $(BROWSER_FILE_DEV)
echo '})();' >> $(BROWSER_FILE_DEV)
Use UglifyJS 2.4.7 The |uglifyjs| call had to be adapted because the options changed significantly in version 2.
11 years ago
$(UGLIFYJS) \
--mangle \
--compress warnings=false \
--comments /Copyright/ \
-o $(BROWSER_FILE_MIN) \
$(BROWSER_FILE_DEV)
Git repo npmization: Make the repo a npm package Includes: * Moving the source code from /src to /lib. * Adding an explicit file list to package.json * Updating the Makefile. * Updating the spec and benchmark suites and their READMEs. Part of a fix for GH-32.
12 years ago
# Remove browser version of the library (created by "browser")
browserclean:
rm -rf $(BROWSER_DIR)
Replace Jakefile with Makefile Doing scripting tasks in JavaScript is painful.
13 years ago
Jasmine: Initial infrastructure This is the first of many commits that gradually convert PEG.js's test suite from QUnit to Jasmine, cleaning it up on the way. Main reason for the change is that Jasmine allows nested contexts, allowing to structure the tests in a better way than QUnit. Moreover, the tests needed to be cleaned up a bit.
13 years ago
# Run the spec suite
Git repo npmization: Make the repo a npm package Includes: * Moving the source code from /src to /lib. * Adding an explicit file list to package.json * Updating the Makefile. * Updating the spec and benchmark suites and their READMEs. Part of a fix for GH-32.
12 years ago
spec:
Jasmine: Initial infrastructure This is the first of many commits that gradually convert PEG.js's test suite from QUnit to Jasmine, cleaning it up on the way. Main reason for the change is that Jasmine allows nested contexts, allowing to structure the tests in a better way than QUnit. Moreover, the tests needed to be cleaned up a bit.
13 years ago
$(JASMINE_NODE) --verbose $(SPEC_DIR)
Replace Jakefile with Makefile Doing scripting tasks in JavaScript is painful.
13 years ago
# Run the benchmark suite
Git repo npmization: Make the repo a npm package Includes: * Moving the source code from /src to /lib. * Adding an explicit file list to package.json * Updating the Makefile. * Updating the spec and benchmark suites and their READMEs. Part of a fix for GH-32.
12 years ago
benchmark:
Replace Jakefile with Makefile Doing scripting tasks in JavaScript is painful.
13 years ago
$(BENCHMARK_RUN)
# Run JSHint on the source
Git repo npmization: Make the repo a npm package Includes: * Moving the source code from /src to /lib. * Adding an explicit file list to package.json * Updating the Makefile. * Updating the spec and benchmark suites and their READMEs. Part of a fix for GH-32.
12 years ago
hint:
Move compiler-related source files and tests into a subdirectory
13 years ago
$(JSHINT) \
Git repo npmization: Make the repo a npm package Includes: * Moving the source code from /src to /lib. * Adding an explicit file list to package.json * Updating the Makefile. * Updating the spec and benchmark suites and their READMEs. Part of a fix for GH-32.
12 years ago
`find $(LIB_DIR) -name '*.js'` \
Jasmine: Initial infrastructure This is the first of many commits that gradually convert PEG.js's test suite from QUnit to Jasmine, cleaning it up on the way. Main reason for the change is that Jasmine allows nested contexts, allowing to structure the tests in a better way than QUnit. Moreover, the tests needed to be cleaned up a bit.
13 years ago
`find $(SPEC_DIR) -name '*.js' -and -not -path '$(SPEC_DIR)/vendor/*'` \
Move compiler-related source files and tests into a subdirectory
13 years ago
$(BENCHMARK_DIR)/*.js \
$(BENCHMARK_RUN) \
Replace Jakefile with Makefile Doing scripting tasks in JavaScript is painful.
13 years ago
$(PEGJS)
Git repo npmization: Make the repo a npm package Includes: * Moving the source code from /src to /lib. * Adding an explicit file list to package.json * Updating the Makefile. * Updating the spec and benchmark suites and their READMEs. Part of a fix for GH-32.
12 years ago
.PHONY: all parser browser browserclean spec benchmark hint
.SILENT: all parser browser browserclean spec benchmark hint