You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

144 lines
6.8 KiB
Makefile

# ===== Variables =====
PEGJS_VERSION = `cat $(VERSION_FILE)`
# ===== Modules =====
# Order matters -- dependencies must be listed before modules dependent on them.
MODULES = utils/arrays \
utils/objects \
utils/classes \
grammar-error \
parser \
compiler/visitor \
compiler/asts \
Code generator rewrite This is a complete rewrite of the PEG.js code generator. Its goals are: 1. Allow optimizing the generated parser code for code size as well as for parsing speed. 2. Prepare ground for future optimizations and big features (like incremental parsing). 2. Replace the old template-based code-generation system with something more lightweight and flexible. 4. General code cleanup (structure, style, variable names, ...). New Architecture ---------------- The new code generator consists of two steps: * Bytecode generator -- produces bytecode for an abstract virtual machine * JavaScript generator -- produces JavaScript code based on the bytecode The abstract virtual machine is stack-based. Originally I wanted to make it register-based, but it turned out that all the code related to it would be more complex and the bytecode itself would be longer (because of explicit register specifications in instructions). The only downsides of the stack-based approach seem to be few small inefficiencies (see e.g. the |NIP| instruction), which seem to be insignificant. The new generator allows optimizing for parsing speed or code size (you can choose using the |optimize| option of the |PEG.buildParser| method or the --optimize/-o option on the command-line). When optimizing for size, the JavaScript generator emits the bytecode together with its constant table and a generic bytecode interpreter. Because the interpreter is small and the bytecode and constant table grow only slowly with size of the grammar, the resulting parser is also small. When optimizing for speed, the JavaScript generator just compiles the bytecode into JavaScript. The generated code is relatively efficient, so the resulting parser is fast. Internal Identifiers -------------------- As a small bonus, all internal identifiers visible to user code in the initializer, actions and predicates are prefixed by |peg$|. This lowers the chance that identifiers in user code will conflict with the ones from PEG.js. It also makes using any internals in user code ugly, which is a good thing. This solves GH-92. Performance ----------- The new code generator improved parsing speed and parser code size significantly. The generated parsers are now: * 39% faster when optimizing for speed * 69% smaller when optimizing for size (without minification) * 31% smaller when optimizing for size (with minification) (Parsing speed was measured using the |benchmark/run| script. Code size was measured by generating parsers for examples in the |examples| directory and adding up the file sizes. Minification was done by |uglify --ascii| in version 1.3.4.) Final Note ---------- This is just a beginning! The new code generator lays a foundation upon which many optimizations and improvements can (and will) be made. Stay tuned :-)
12 years ago
compiler/opcodes \
compiler/javascript \
Code generator rewrite This is a complete rewrite of the PEG.js code generator. Its goals are: 1. Allow optimizing the generated parser code for code size as well as for parsing speed. 2. Prepare ground for future optimizations and big features (like incremental parsing). 2. Replace the old template-based code-generation system with something more lightweight and flexible. 4. General code cleanup (structure, style, variable names, ...). New Architecture ---------------- The new code generator consists of two steps: * Bytecode generator -- produces bytecode for an abstract virtual machine * JavaScript generator -- produces JavaScript code based on the bytecode The abstract virtual machine is stack-based. Originally I wanted to make it register-based, but it turned out that all the code related to it would be more complex and the bytecode itself would be longer (because of explicit register specifications in instructions). The only downsides of the stack-based approach seem to be few small inefficiencies (see e.g. the |NIP| instruction), which seem to be insignificant. The new generator allows optimizing for parsing speed or code size (you can choose using the |optimize| option of the |PEG.buildParser| method or the --optimize/-o option on the command-line). When optimizing for size, the JavaScript generator emits the bytecode together with its constant table and a generic bytecode interpreter. Because the interpreter is small and the bytecode and constant table grow only slowly with size of the grammar, the resulting parser is also small. When optimizing for speed, the JavaScript generator just compiles the bytecode into JavaScript. The generated code is relatively efficient, so the resulting parser is fast. Internal Identifiers -------------------- As a small bonus, all internal identifiers visible to user code in the initializer, actions and predicates are prefixed by |peg$|. This lowers the chance that identifiers in user code will conflict with the ones from PEG.js. It also makes using any internals in user code ugly, which is a good thing. This solves GH-92. Performance ----------- The new code generator improved parsing speed and parser code size significantly. The generated parsers are now: * 39% faster when optimizing for speed * 69% smaller when optimizing for size (without minification) * 31% smaller when optimizing for size (with minification) (Parsing speed was measured using the |benchmark/run| script. Code size was measured by generating parsers for examples in the |examples| directory and adding up the file sizes. Minification was done by |uglify --ascii| in version 1.3.4.) Final Note ---------- This is just a beginning! The new code generator lays a foundation upon which many optimizations and improvements can (and will) be made. Stay tuned :-)
12 years ago
compiler/passes/generate-bytecode \
compiler/passes/generate-javascript \
compiler/passes/remove-proxy-rules \
compiler/passes/report-left-recursion \
compiler/passes/report-missing-rules \
compiler \
peg
# ===== Directories =====
SRC_DIR = src
LIB_DIR = lib
BIN_DIR = bin
BROWSER_DIR = browser
SPEC_DIR = spec
BENCHMARK_DIR = benchmark
NODE_MODULES_DIR = node_modules
NODE_MODULES_BIN_DIR = $(NODE_MODULES_DIR)/.bin
# ===== Files =====
PARSER_SRC_FILE = $(SRC_DIR)/parser.pegjs
PARSER_OUT_FILE = $(LIB_DIR)/parser.js
BROWSER_FILE_DEV = $(BROWSER_DIR)/peg-$(PEGJS_VERSION).js
BROWSER_FILE_MIN = $(BROWSER_DIR)/peg-$(PEGJS_VERSION).min.js
VERSION_FILE = VERSION
# ===== Executables =====
JSHINT = $(NODE_MODULES_BIN_DIR)/jshint
UGLIFYJS = $(NODE_MODULES_BIN_DIR)/uglifyjs
JASMINE_NODE = $(NODE_MODULES_BIN_DIR)/jasmine-node
PEGJS = $(BIN_DIR)/pegjs
BENCHMARK_RUN = $(BENCHMARK_DIR)/run
# ===== Targets =====
# Default target
all: browser
# Generate the grammar parser
parser:
$(PEGJS) $(PARSER_SRC_FILE) $(PARSER_OUT_FILE)
# Build the browser version of the library
browser:
mkdir -p $(BROWSER_DIR)
rm -f $(BROWSER_FILE_DEV)
rm -f $(BROWSER_FILE_MIN)
# The following code is inspired by CoffeeScript's Cakefile.
echo '/*' >> $(BROWSER_FILE_DEV)
echo " * PEG.js $(PEGJS_VERSION)" >> $(BROWSER_FILE_DEV)
echo ' *' >> $(BROWSER_FILE_DEV)
echo ' * http://pegjs.org/' >> $(BROWSER_FILE_DEV)
echo ' *' >> $(BROWSER_FILE_DEV)
echo ' * Copyright (c) 2010-2013 David Majda' >> $(BROWSER_FILE_DEV)
echo ' * Licensed under the MIT license.' >> $(BROWSER_FILE_DEV)
echo ' */' >> $(BROWSER_FILE_DEV)
echo 'var PEG = (function(undefined) {' >> $(BROWSER_FILE_DEV)
echo ' var modules = {' >> $(BROWSER_FILE_DEV)
echo ' define: function(name, factory) {' >> $(BROWSER_FILE_DEV)
echo ' var dir = name.replace(/(^|\/)[^/]+$$/, "$$1"),' >> $(BROWSER_FILE_DEV)
echo ' module = { exports: {} };' >> $(BROWSER_FILE_DEV)
echo '' >> $(BROWSER_FILE_DEV)
echo ' function require(path) {' >> $(BROWSER_FILE_DEV)
echo ' var name = dir + path,' >> $(BROWSER_FILE_DEV)
echo ' regexp = /[^\/]+\/\.\.\/|\.\//;' >> $(BROWSER_FILE_DEV)
echo '' >> $(BROWSER_FILE_DEV)
echo " /* Can't use /.../g because we can move backwards in the string. */" >> $(BROWSER_FILE_DEV)
echo ' while (regexp.test(name)) {' >> $(BROWSER_FILE_DEV)
echo ' name = name.replace(regexp, "");' >> $(BROWSER_FILE_DEV)
echo ' }' >> $(BROWSER_FILE_DEV)
echo '' >> $(BROWSER_FILE_DEV)
echo ' return modules[name];' >> $(BROWSER_FILE_DEV)
echo ' }' >> $(BROWSER_FILE_DEV)
echo '' >> $(BROWSER_FILE_DEV)
echo ' factory(module, require);' >> $(BROWSER_FILE_DEV)
echo ' this[name] = module.exports;' >> $(BROWSER_FILE_DEV)
echo ' }' >> $(BROWSER_FILE_DEV)
echo ' };' >> $(BROWSER_FILE_DEV)
echo '' >> $(BROWSER_FILE_DEV)
for module in $(MODULES); do \
echo " modules.define(\"$$module\", function(module, require) {" >> $(BROWSER_FILE_DEV); \
sed -e 's/^/ /' lib/$$module.js >> $(BROWSER_FILE_DEV); \
echo ' });' >> $(BROWSER_FILE_DEV); \
echo '' >> $(BROWSER_FILE_DEV); \
done
echo ' return modules["peg"]' >> $(BROWSER_FILE_DEV)
echo '})();' >> $(BROWSER_FILE_DEV)
$(UGLIFYJS) \
--mangle \
--compress warnings=false \
--comments /Copyright/ \
-o $(BROWSER_FILE_MIN) \
$(BROWSER_FILE_DEV)
# Remove browser version of the library (created by "browser")
browserclean:
rm -rf $(BROWSER_DIR)
# Run the spec suite
spec:
$(JASMINE_NODE) --verbose $(SPEC_DIR)
# Run the benchmark suite
benchmark:
$(BENCHMARK_RUN)
# Run JSHint on the source
hint:
$(JSHINT) \
`find $(LIB_DIR) -name '*.js'` \
`find $(SPEC_DIR) -name '*.js' -and -not -path '$(SPEC_DIR)/vendor/*'` \
$(BENCHMARK_DIR)/*.js \
$(BENCHMARK_RUN) \
$(PEGJS)
.PHONY: all parser browser browserclean spec benchmark hint
.SILENT: all parser browser browserclean spec benchmark hint