Initial commit; v1.0.0

master
Sven Slootweg 8 years ago
commit e5bce45095

4
.gitignore vendored

@ -0,0 +1,4 @@
/node_modules/
/lib/
test-config.json
errors

@ -0,0 +1,3 @@
/node_modules/
test-config.json
errors

@ -0,0 +1,3 @@
## 1.0.0 (November 18, 2016)
Initial release.

@ -0,0 +1,181 @@
# report-errors
An easy-to-use tool for tracking unhandled errors.
* Automatically and immediately e-mails you when an unhandled error occurs in your application.
* Easy to set up.
* Supports both synchronous and asynchronous errors (using Promises).
* Includes both a simplified stacktrace and full error details in the e-mail.
* Supports manual reporting of errors (eg. from error-handling middleware).
* Automatically crashes the process after reporting an error, to prevent data loss or corruption.
* Reporting is handled from *outside* of your application process, using a dedicated daemon - this minimizes the chance of data loss.
* Configurable e-mail subject line.
* Works with any SMTP provider, as well as directly from the server (but read the caveat below).
An example of an error report:
![Screenshot of report e-mail](https://git.cryto.net/joepie91/node-report-errors/raw/master/screenshot.png)
## Crashing processes and safe error handling
By default, `report-error` will crash your process once it has encountered and reported an error, in the expectation that a service manager will restart it. While this behaviour can be disabled, __I strongly recommend against that__.
When an unhandled error occurs, that means that your application was not aware of how to handle this error - thus, it also cannot know what application state was affected by this error. The application is now in an *undefined state*, and continuing to run in an undefined state could lead to __data loss, corruption, or security issues__.
The only safe thing to do after encountering an unhandled error, is for your application to crash as soon as possible, and be restarted cleanly. For this reason, you should leave the default behaviour intact if at all possible, and rely on your service manager (systemd, `forever`, PM2, etc.) to restart the application.
If you are concerned about downtime, you can run your application in [cluster mode](https://nodejs.org/api/cluster.html), or as multiple processes behind a load balancer. This way, if a single process crashes, new requests will simply be redirected to other processes while your crashed process is restarting.
## Sending reports without an SMTP provider
While `report-error` can send reports directly to a specified e-mail address without using an external SMTP server, this is *not recommended* in most cases. Spamfilters will generally distrust these kind of "directly-sent" e-mails, for a variety of reasons. You *can* make it work by explicitly setting a "this is not spam" rule on the receiving side, but it's generally easier to just use an SMTP provider of some sort.
## License
[WTFPL](http://www.wtfpl.net/txt/copying/) or [CC0](https://creativecommons.org/publicdomain/zero/1.0/), whichever you prefer. A donation and/or attribution are appreciated, but not required.
## Donate
Maintaining open-source projects takes a lot of time, and the more donations I receive, the more time I can dedicate to open-source. If this module is useful to you, consider [making a donation](http://cryto.net/~joepie91/donate.html)!
You can donate using Bitcoin, PayPal, Flattr, cash-in-mail, SEPA transfers, and pretty much anything else. Thank you!
## Contributing
Pull requests welcome. Please make sure your modifications are in line with the overall code style, and ensure that you're editing the files in `src/`, not those in `lib/`.
Build tool of choice is `gulp`; simply run `gulp` while developing, and it will watch for changes.
Be aware that by making a pull request, you agree to release your modifications under the licenses stated above.
## Usage
This library consists of two parts:
1. The error handling library
2. The reporting daemon
The error handling library is purely responsible for catching all unhandled errors in the application, dumping them to a JSON file on disk, and subsequently crashing the process. It does not attempt to do any formatting or post-processing. The reporting daemon picks up the dumped files (by watching the filesystem), formats them into e-mails with attachments, and sends them off.
### Creating a configuration file
The reporting daemon uses a single configuration file to determine where to send reports. A typical configuration file might look something like this:
```json
{
"errorPath": "/opt/my-project/errors",
"stackFilter": "*",
"metadata": {
"from": "ops@cryto.net",
"to": "admin@cryto.net"
}
}
```
For the sake of this documentation, we'll assume that you've saved this file as `reporter-config.json`, but you can pick any filename - you'll have to provide its path explicitly when you run the error reporter binary, either way.
Valid options are:
* __errorPath:__ The directory that your errors are stored in. You will configure this later in the error handling library as well. This can be any path, but generally you'll want it to be an `errors` subdirectory in your project directory.
* __stackFilter:__ What modules to filter out of the simplified stacktraces shown in the e-mail report. This can either be the string `"*"` (to filter out *every* third-party module), or an array of module names to filter. Note that the e-mail will always include a JSON attachment containing the *full* stacktrace - this setting purely affects the e-mail body.
* __subjectFormat:__ The format for the subject line of the report e-mail. In this string, `$type` will be replaced with the error type/name, and `$message` will be replaced with the error message.
* __metadata:__
* __from:__ The sender address displayed on the e-mail report.
* __to:__ The address to e-mail reports to.
* __smtp:__ *Optional.* The SMTP server to use for sending the e-mail reports. If not configured, e-mails are delivered directly to the recipient's e-mail server (but see the caveat above).
* __hostname:__ The hostname on which the SMTP server can be reached.
* __port:__ The port number that the SMTP server is accessible on.
* __username:__ Your username for the SMTP server.
* __password:__ Your password for the SMTP server.
* __nodemailer:__ *All optional.* Custom options to be passed directly into the underlying `nodemailer` instance.
* __secure:__ Forces SSL/TLS usage.
* __requireTLS:__ Forces STARTTLS usage.
* __ignoreTLS:__ Prevents usage of either SSL/TLS or STARTTLS.
* __tls:__ Custom TLS options to pass to the [`tls` core library](https://nodejs.org/api/tls.html#tls_class_tls_tlssocket).
* __localAddress:__ The local network interface to use for network connections.
* __connectionTimeout:__ How many milliseconds to wait for a connection to be established.
* __greetingTimeout:__ How many milliseconds to wait for the SMTP greeting to be received.
* __socketTimeout:__ How many milliseconds of inactivity to allow, before a connection is closed.
### Setting up the daemon
The exact setup instructions will vary depending on what service manager you use, but you should ensure that the following command is run somehow:
```sh
/path/to/your/project/node_modules/.bin/report-errors /path/to/your/reporter-config.json
```
That's it! The daemon will watch the configured error directory, and e-mail you any errors that appear. Make sure that your service manager is configured to restart the process if it crashes.
You will probably also want to configure your service manager to log the output of the application - if something breaks in the reporting daemon, it will be printed to the terminal. It can't e-mail you the error if the reporter itself is broken!
### Setting up the library
Using the library is pretty simple - simply add the following at the start of your application's entry point:
```javascript
const path = require("path");
const reportErrors = require("report-errors");
// ...
let errorReporter = reportErrors(path.join(__dirname, "errors"));
```
This will store all errors in the `errors` subdirectory relative to your entry point (which is usually, but not always, the root of your project). You should ensure that your reporter configuration file has this same path set as its `errorPath`.
In some cases, you might want to report unhandled errors manually - for example, from Express error handling middleware. You'd use the `report` method for that, like so:
```javascript
/* Assuming `app` contains an Express application, and the `errorReporter` has been defined as before... */
app.use((error, req, res, next) => {
if (error.statusCode != null && error.statusCode >= 400 && error.statusCode < 500) {
/* This is a client error, such as a 401 or a 403. */
res.status(error.statusCode).send(error.message);
} else {
/* This is some other kind of error we didn't expect. */
errorReporter.report(error, {
req: req,
res: res
});
}
})
```
Note how we're not just passing in the `error`, but also an additional object containing `req` and `res` - this object is the *context* object, and it can contain any JSON-serializable data. It's stored alongside the error, and it can be useful to more easily reproduce an error - for example, in this case, the serialized version of the `req` object will include things like the request method and path.
## Building your own error processing tools
While the default error reporting daemon is good enough for most simple deployments, in some cases you may want to develop your own tooling for dealing with the reported errors. All `report-errors` errors are stored on disk in a standardized JSON format:
* __error:__ The serialized error object. Usually includes at least `name`, `message`, `code`, and `stack` (as well as custom properties), but none of these are guaranteed to exist. The stacktrace is the original string-formatted stacktrace, and it's up to you to parse it as needed.
* __environmentName:__ A brief description of the environment that the error occurred in - operating system, runtime, and so on. As provided by the [`env-name`](https://github.com/vdemedes/env-name) module.
* __environmentInfo:__ A structured set of information about the environment that the error occurred in. As provided by the [`env-info`](https://github.com/vdemedes/env-info) module.
* __hostname:__ The hostname of the system on which the error occurred.
* __context:__ An object containing the "context" that was passed in with the reported error. For an uncaught rejection, this will contain the originating Promise as a `promise` property, but the `context` object may contain *any* kind of context that the reporting code felt was necessary to reproduce the error. This information is meant for later human inspection.
The JSON attachments in the default reporter's e-mails include the above properties, but also several additional properties:
* __parsedStack:__ A parsed version of the stacktrace, as produced by the [`stacktrace-parser`](https://www.npmjs.com/package/stacktrace-parser) module.
* __simplifiedStack:__ A *simplified* version of the parsed stacktrace, simplified according to the `stackFilter` option in your reporter configuration. All removed stacktrace items are replaced by an object of the format `{type: "ignored", count: 9}`, where the `count` property indicates *how many* stacktrace items were omitted.
## API
### reportErrors(errorPath, [options])
Creates a new `errorReporter` instance.
* __errorPath:__ Path where all errors will be stored. This must be writable by the application, and it must be the same path that the configuration file for the reporting daemon points at.
* __options:__
* __doNotCrash:__ If set to `true`, it prevents the library from crashing your process after an error is reported. This is *extremely dangerous*, and you should only use it if you are fully aware of the consequences. Defaults to `false`.
__WARNING:__ Note that *as soon as you create an instance*, it will start intercepting errors, and things like the default `uncaughtException` handler will no longer fire!
### errorReporter.report(error, [context])
Manually reports an `error` with the given `context`. It will be treated the same as any automatically caught errors.
* __error:__ The error to report. *Must* be an Error object or descendant thereof.
* __context:__ Any data associated with the context in which the error occurred. For example, when reporting an unhandled error from your Express error-handling middleware, you might want to include `req` and `res` here. There is no standard set of keys/values to use here - include whatever information you feel is important to reproduce the error, as long as it's JSON-serializable; circular references are fine.

@ -0,0 +1,18 @@
var gulp = require("gulp");
var presetES2015 = require("@joepie91/gulp-preset-es2015");
var source = ["src/**/*.js"]
gulp.task('babel', function() {
return gulp.src(source)
.pipe(presetES2015({
basePath: __dirname
}))
.pipe(gulp.dest("lib/"));
});
gulp.task("watch", function () {
gulp.watch(source, ["babel"]);
});
gulp.task("default", ["babel", "watch"]);

@ -0,0 +1,3 @@
'use strict';
module.exports = require("./lib/library");

@ -0,0 +1,43 @@
{
"name": "report-errors",
"version": "1.0.0",
"description": "A tool for reporting unexpected errors in your application by e-mail",
"main": "index.js",
"bin": {
"report-errors": "./lib/daemon/index.js"
},
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"repository": {
"type": "git",
"url": "http://git.cryto.net/joepie91/node-report-errors.git"
},
"keywords": [
"monitoring",
"email",
"error handling",
"errors",
"promises"
],
"author": "Sven Slootweg",
"license": "WTFPL",
"dependencies": {
"bluebird": "^3.3.3",
"chokidar": "^1.6.1",
"env-info": "^1.0.0",
"env-name": "^1.0.0",
"json-stringify-safe": "^5.0.1",
"nodemailer": "^2.6.4",
"object.omit": "^2.0.1",
"object.pick": "^1.2.0",
"stacktrace-parser": "^0.1.4",
"unhandled-error": "^1.0.0"
},
"devDependencies": {
"@joepie91/gulp-preset-es2015": "^1.0.1",
"babel-preset-es2015": "^6.6.0",
"create-error": "^0.3.1",
"gulp": "^3.9.1"
}
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 36 KiB

@ -0,0 +1,41 @@
'use strict';
const nodemailer = require("nodemailer");
const objectPick = require("object.pick");
function getNodemailerOptions(config) {
if (config.nodemailer == null) {
return {};
} else {
return objectPick(config.nodemailer, [
"secure", "ignoreTLS", "requireTLS", "tls",
"localAddress", "connectionTimeout", "greetingTimeout", "socketTimeout"
]);
}
}
function getTransportOptions(config, nodemailerOptions) {
if (config.smtp == null) {
console.error("WARNING: Sending e-mails directly to the recipient's mailserver. If you are not explicitly whitelisting the source, your notifications will most likely end up in Spam. Add an SMTP relay to your configuration file to avoid this issue.");
return Object.assign(nodemailerOptions, {
direct: true
});
} else {
return Object.assign(nodemailerOptions, {
host: config.smtp.hostname,
port: config.smtp.port,
auth: {
user: config.smtp.username,
pass: config.smtp.password
}
});
}
}
module.exports = function createEmailSender(config) {
let nodemailerOptions = getNodemailerOptions(config);
let transportOptions = getTransportOptions(config, nodemailerOptions);
return nodemailer.createTransport(transportOptions, config.metadata);
};

@ -0,0 +1,20 @@
'use strict';
const prettifyStack = require("../error/stack/prettify");
module.exports = function generateEmailPreview({environmentName, error, hostname}, stackFilter) {
return `
An unhandled ${error.name} occurred.
Environment: ${environmentName}
Hostname: ${hostname}
__________________________________________
Simplified error stacktrace:
${prettifyStack(error.stackWithout(stackFilter, true))}
__________________________________________
The full error information is attached.
`.trim();
};

@ -0,0 +1,34 @@
'use strict';
const stacktraceParser = require("stacktrace-parser");
const objectOmit = require("object.omit");
const printError = require("../print-error");
const parseStacktrace = require("./stack/parse");
const filterModules = require("./stack/filter-modules");
const filterAllModules = require("./stack/filter-all-modules");
const collapseIgnored = require("./stack/collapse-ignored");
module.exports = function parseError(error) {
return {
name: error.name,
message: error.message,
stack: parseStacktrace(error.stack),
properties: objectOmit(error, ["name", "message", "stack"]),
stackWithout: function stackWithout(modules, collapse = false) {
let filteredStack;
if (modules === "*") {
filteredStack = filterAllModules(this.stack);
} else {
filteredStack = filterModules(this.stack, modules);
}
if (collapse) {
return collapseIgnored(filteredStack);
} else {
return filteredStack;
}
}
}
};

@ -0,0 +1,23 @@
'use strict';
const Promise = require("bluebird");
const fs = Promise.promisifyAll(require("fs"));
let maxAttempts = 100;
module.exports = function readError(errorFilePath, attempt = 0) {
return Promise.try(() => {
return fs.readFileAsync(errorFilePath);
}).then((fileContents) => {
return JSON.parse(fileContents);
}).catch(SyntaxError, (err) => {
if (attempt < maxAttempts) {
/* The JSON hasn't been completely written yet. We'll retry in a bit. */
return Promise.delay(1000).then(() => {
reportError(errorFilePath, attempt + 1);
});
} else {
throw new Error(`Could not parse error file ${errorFilePath}, and reached maximum attempts`);
}
});
};

@ -0,0 +1,21 @@
'use strict';
module.exports = function collapseIgnored(stack) {
let sawIgnored = false;
return stack.reduce((newStack, stackLine) => {
if (stackLine.type === "ignored") {
if (sawIgnored === false) {
sawIgnored = true;
newStack.push(stackLine);
} else {
newStack[newStack.length - 1].count += 1;
}
} else {
sawIgnored = false;
newStack.push(stackLine);
}
return newStack;
}, []);
};

@ -0,0 +1,16 @@
'use strict';
module.exports = function filterAllModules(stack) {
return stack.map((stackLine) => {
if (stackLine.file.includes("/node_modules/") || stackLine.file.includes("\\node_modules\\") || (!stackLine.file.includes("/") && !stackLine.file.includes("\\"))) {
return {
type: "ignored",
count: 1
}
} else {
return Object.assign({
type: "stack"
}, stackLine);
}
});
};

@ -0,0 +1,22 @@
'use strict';
const parseModules = require("./parse-modules");
module.exports = function filterModules(stack, ignoredModules) {
let newStack = [];
return stack.map((stackLine) => {
let moduleList = parseModules(stackLine.file);
if (moduleList.some(module => ignoredModules.includes(module))) {
return {
type: "ignored",
count: 1
}
} else {
return Object.assign({
type: "stack"
}, stackLine);
}
});
};

@ -0,0 +1,21 @@
'use strict';
module.exports = function parseModules(filePath) {
if (!filePath.includes("/") && !filePath.includes("\\")) {
return ["<internal>"];
} else {
let regex = /(?:\/|\\)node_modules(?:\/|\\)([^\/\\]+)/g;
let match, matches = [];
while (match = regex.exec(filePath)) {
matches.push(match);
}
console.log(filePath)
console.log(matches)
return matches.map((match) => {
return match[1];
});
}
};

@ -0,0 +1,19 @@
'use strict';
const stacktraceParser = require("stacktrace-parser");
const printError = require("../../print-error");
module.exports = function parseStacktrace(stacktraceString) {
try {
return stacktraceParser.parse(stacktraceString);
} catch (err) {
printError([
"WARNING: Unable to parse stacktrace!",
err,
"Original stacktrace string:",
stacktraceString
]);
return [];
}
};

@ -0,0 +1,11 @@
'use strict';
module.exports = function prettifyStack(stack) {
return stack.map((stackLine) => {
if (stackLine.type === "ignored") {
return ` ... omitted ${stackLine.count} lines ...`;
} else {
return `${stackLine.methodName} (line ${stackLine.lineNumber} in ${stackLine.file})`
}
}).join("\n");
};

@ -0,0 +1,64 @@
#!/usr/bin/env node
'use strict';
const Promise = require("bluebird");
const chokidar = require("chokidar");
const fs = Promise.promisifyAll(require("fs"));
const path = require("path");
const stacktraceParser = require("stacktrace-parser");
const loadConfiguration = require("./load-configuration");
const printError = require("./print-error");
const createEmailSender = require("./email/create-sender");
const generateEmailPreview = require("./email/generate-preview");
const readError = require("./error/read");
const parseError = require("./error/parse");
let config = loadConfiguration(process.argv[2]);
let transport = createEmailSender(config);
function reportError(errorFilePath) {
Promise.try(() => {
return readError(errorFilePath);
}).then((errorData) => {
let error = parseError(errorData.error);
// FIXME: Strip everything after the first line of the message?
let subjectLine = config.subjectFormat.replace(/\$type/g, error.name).replace(/\$message/g, error.message.replace(/\n/g, " "));
return Promise.try(() => {
return transport.sendMail({
subject: subjectLine,
text: generateEmailPreview({
environmentName: errorData.environmentName,
hostname: errorData.hostname,
error: error,
}, config.stackFilter),
attachments: [{
filename: path.basename(errorFilePath),
content: JSON.stringify(Object.assign(errorData, {
parsedStack: error.stack,
simplifiedStack: error.stackWithout("*", true)
}), null, "\t")
}]
});
}).then(() => {
console.log(`Reported error in ${path.basename(errorFilePath)}: ${error.name} - ${error.message}`);
});
}).catch((err) => {
printError([
`ERROR: Unable to parse or report error for ${errorFilePath}.`,
err
]);
});
}
chokidar.watch(config.errorPath, {ignoreInitial: true})
.on("add", (errorFilePath) => {
reportError(errorFilePath);
})
.on("ready", () => {
console.log("Listening for new errors...");
});

@ -0,0 +1,32 @@
'use strict';
const fs = require("fs");
module.exports = function loadConfiguration(configurationPath) {
if (configurationPath == null) {
console.error("You must specify a configuration file.");
process.exit(1);
}
let config = JSON.parse(fs.readFileSync(configurationPath));
if (config.metadata == null || config.metadata.from == null || config.metadata.to == null) {
console.error("Your configuration file must specify at least a sender (metadata.from) and a recipient (metadata.to).");
process.exit(1);
}
if (config.errorPath == null) {
console.error("Your configuration file must specify an errorPath.");
process.exit(1);
}
if (config.subjectFormat == null) {
config.subjectFormat = "UNHANDLED ERROR: $type - $message"
}
if (config.stackFilter == null) {
config.stackFilter = "*";
}
return config;
};

@ -0,0 +1,13 @@
'use strict';
module.exports = function printError(errorLines) {
console.error("_______________________________________");
console.error("");
errorLines.forEach((line) => {
console.error(line);
});
console.error("_______________________________________");
console.error("");
};

@ -0,0 +1,37 @@
'use strict';
const fs = require("fs");
const os = require("os");
const path = require("path");
const unhandledError = require("unhandled-error");
const jsonStringifySafe = require("json-stringify-safe");
const envName = require("env-name");
const envInfo = require("env-info");
const serializeError = require("./serialize-error");
module.exports = function createErrorReporter(errorPath, options = {}) {
let unhandledErrorHandler = unhandledError((error, context) => {
console.log(context);
let stringifiedErrorData = jsonStringifySafe({
error: serializeError(error),
context: context,
environmentName: envName(),
environmentInfo: envInfo(),
hostname: os.hostname()
}, undefined, "\t", () => {});
let errorFilename = `${Date.now()}_${Math.floor(Math.random() * 100000)}.json`;
fs.writeFileSync(path.join(errorPath, errorFilename), stringifiedErrorData);
if (options.handler != null) {
options.handler(error, context);
}
}, {doNotCrash: options.doNotCrash});
return {
report: unhandledErrorHandler.report.bind(unhandledErrorHandler)
}
}

@ -0,0 +1,10 @@
'use strict';
module.exports = function serializeError(error) {
/* The `name`, `message` and `stack` properties are not always enumerable, so we need to add them explicitly. */
return Object.assign({}, error, {
stack: error.stack,
message: error.message,
name: error.name
});
};

@ -0,0 +1,26 @@
'use strict';
const Promise = require("bluebird");
const path = require("path");
const createError = require("create-error");
const reportErrors = require("./");
let TestingError = createError("TestingError", {
someProperty: "foo"
});
let errorReporter = reportErrors(path.join(__dirname, "errors"));
setTimeout(() => {
console.log("bar");
}, 500);
Promise.try(() => {
console.log("foo");
throw new TestingError("Request limit exceeded, API is now on fire", {
qux: {
quz: 1,
quack: [2, false]
}
});
});
Loading…
Cancel
Save