p This is a simple streaming (WebSocket) API, providing a live feed of (public) pastes as they are submitted to <a href="http://pastebin.com/">Pastebin.com</a>. You can find the source code <a href="https://git.cryto.net/joepie91/pastebin-stream">here</a>.
p Since Pastebin does not offer a streaming API itself, this API works by regularly polling the Pastebin API for new posts. This means that posts may be delayed by about a minute, and that they'll be broadcast in 'batches'.
| Resolved a bug that produced an unexpected error when a client sent invalid data.
p Note that the application kills and restarts itself as a safety precaution when an unexpected error is encountered; if you experienced crashes while trying to use the API yesterday, those issues should now be resolved.
p None. But this is a non-commercial and free service, so please be reasonable, and consider <a href="#donate">donating</a> if this service helped you out in some way!
h2 Can I use this from a browser?
p Yes. Cross-domain connections are allowed. But again, be reasonable - if you expect to be making hundreds or thousands of simultaneous connections, please run your own proxy.
a(name="donate")
h2 Donate
p If you like this service, help keep it running by making a donation!
input(type="image", src="https://www.paypalobjects.com/en_US/i/btn/btn_donate_LG.gif", border="0", name="submit", alt="PayPal, de veilige en complete manier van online betalen.")
input(type="image", src="https://www.paypalobjects.com/en_US/i/btn/btn_donate_LG.gif", border="0", name="submit", alt="PayPal, de veilige en complete manier van online betalen.")
p Please send your donation to <strong>13quzALQ98dpJLPVcVA1rbEbE9Brf6VKyc</strong> :)
h2 Future plans
p In the future, I will likely add support for other Pastebin services as well. Suggestions are welcome - send me an e-mail!
h2 Contact
p If you need to reach me for some reason, you can e-mail me at <a href="admin@cryto.net">admin@cryto.net</a>.
h2 API documentation
p This API exposes a single endpoint, at <code>wss://pastebin-stream.cryto.net/stream</code>. Every command below is meant to be sent through that connection. The server expects well-formed JSON, and will kill a connection if it sends malformed data.
p No special clients are required; you can simply use the standard <a href="https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API/Writing_WebSocket_client_applications">WebSocket API</a> in browsers. For Node.js, I recommend using <a href="https://www.npmjs.com/package/ws">the <code>ws</code> library</a>. For other languages, simply use whichever WebSocket implementation is available to you.
p The server sends out a 'ping' roughly every 5 seconds. If you wish to detect disconnections, this is what you'll want to check for.
h3 Making sure you don't miss anything
p The server maintains a backlog of pastes. The exact amount is subject to change, but assume "several hundred". You can make a backlog request to obtain these pastes from the backlog.
p When connecting for the first time, you should do the following to obtain the maximum amount of pastes and make sure you're not left with a gap:
ol
li Set up a message handler.
li Send a subscribe request to receive <code>newPaste</code> events (immediately storing pastes once they start coming in).
li Request a backlog <em>after</em> sending the subscribe request (either full or partial).
li Once the backlog is returned, remove any pastes that you already received through <code>newPaste</code> events, and then insert the backlog <em>before</em> the already-received pastes.
p The API also includes a <code>counter</code> property with every paste. This is an incrementing counter managed by the server, that can be used to obtain missed pastes after a reconnect.
p If you've detected a connection loss and initiated a new connection, you should do the following to obtain all the pastes you missed:
ol
li Set up a message handler.
li Send a subscribe request to receive <code>newPaste</code> events (immediately storing pastes once they start coming in).
li Request a backlog <em>after</em> sending the subscribe request, <strong>specifying the <code>since</code> parameter</strong> - it should contain the <code>counter</code> of the last paste you've received on the previous connection.
li Once the backlog is returned, remove any pastes that you already received through <code>newPaste</code> events, and then insert the backlog <em>before</em> the already-received pastes.
p Note that when the server restarts, <strong>the <code>counter</code> values will be reset to zero</strong>. This means that a given <code>counter</code> value is only unique until the next server restart.
h3 Paste format
p A paste object looks like the following:
pre.
{
"counter": 521,
"service": "pastebinCom",
"id": "V2rpgLwN",
"title": "Paste Title Goes Here",
"date": 1489876199,
"expiry": 0,
"language": "text",
"url": "http://pastebin.com/gPifdLrb"
"contents": "[ ... paste contents go here ... ]"
}
p An explanation of each of the properties:
ul
li <strong>counter</strong>: Unique incrementing counter. Used for catching up on missed pastes.
li <strong>service</strong>: The pastebin service that this paste originated from. Currently, <code>pastebinCom</code> is the only option here.
li <strong>id</strong>: The identifier of the paste, as specified by the pastebin service. The meaning of this is service-specific, and there is no guarantee that it will be unique between services.
li <strong>title</strong>: <em>Optional.</em> The title given to the paste.
li <strong>date</strong>: <em>Optional.</em> A UNIX timestamp (in seconds) indicating when the paste was originally posted.
li <strong>expiry</strong>: <em>Optional.</em> The time <em>in seconds</em> after which the paste will expire (relative to the original posting date).
li <strong>language</strong>: <em>Optional.</em> The language that the user picked for this paste. The values are service-specific.
li <strong>username</strong>: <em>Optional.</em> The username of the user that posted the paste.
li <strong>url</strong>: The original URL for viewing the paste.
li <strong>contents</strong>: The raw contents of the paste, as a string.
p Any <em>optional</em> properties may be missing. You should take this into account in your implementation.
h3 Subscribing to pastes
pre.
{
"type": "subscribe"
}
p You should send this immediately after connecting. Once you do, you will start receiving pastes. Incoming pastes look like the following:
pre.
{
"type": "newPaste",
"data": <a single paste object goes here>
}
h3 Requesting a backlog
p There are a few options for request a backlog, but the <em>response</em> is always in the same format:
pre.
{
"type": "backlog",
"results": [
... an array of paste objects goes here ...
]
}
h4 Requesting the full backlog
p This retrieves every single paste that the server has in its backlog. Note that the response may be large.
pre.
{
"type": "backlog",
"all": true
}
h4 Requesting the last X entries
p This retrieves only the last X entries in the backlog, where X is an amount you specify.
pre.
{
"type": "backlog",
"last": 25
}
h4 Requesting the backlog since a specific paste
p This retrieves every paste in the backlog since a given <code>counter</code> value.
