You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
Sven Slootweg c35a80c7b3 Fix backpressure handling, improve logging, export explicit fromTransform 2 years ago
src Fix backpressure handling, improve logging, export explicit fromTransform 2 years ago
.gitignore Initial commit 4 years ago
README.md Flotsam and jetsam 3 years ago
example.js Ensure that a writable stream doesn't end until the underlying stream has ended, plus misc changes 4 years ago
index.js Fix backpressure handling, improve logging, export explicit fromTransform 2 years ago
numbers.txt Initial commit 4 years ago
package.json Fix backpressure handling, improve logging, export explicit fromTransform 2 years ago
yarn.lock Fix backpressure handling, improve logging, export explicit fromTransform 2 years ago

README.md

TODO

Special case: Duplex streams

In Node.js streams, there is a thing called a Duplex stream. This is basically a stream that has an independent readable and writable part, that are not connected together. Like a TCP socket, for example; you send data through the writable half and receive data through the readable half - but the data you receive is not the same data as you sent.

Duplex streams are really just two independent streams that are stuffed into a single object, representing a single resource, like a network socket. And this is how ppstreams treats those - as two independent streams. In Promistreams, there are no Duplex streams, so when converting a Node.js Duplex stream, you will have to do the conversion twice - once for the readable part, and once for the writable part. This can be done with the duplexRead and duplexWrite methods.

Why does Promistreams not have Duplex streams? Because they're surprisingly tricky to implement. They would make the Promistreams specification a lot more complex, and make certain things impossible; for example, a sink stream (writable stream) in Promistreams can return a Promise to signify when it's completely done receiving data, but a source stream (readable stream) needs to return a Promise with the next read value! You can only return one or the other, so supporting both would involve a lot of special logic for detecting what kind of thing is being returned.

Aside from that, they are also just difficult to work with. Developers routinely get tripped up by the difference between a Duplex and a Transform stream, and it's much simpler to reason about your streams code when a separate readable and writable stream don't pretend to be a single stream when they're really not.

--

Things to document:

  • "Cannot read from a partial pipeline; maybe you forgot to specify a source stream?" means that the stream you thought was readable/writable is actually a transform stream that's internally wired up to something else, and you should use the explicit fromReadable/fromWritable methods in that case. The adapter library cannot (currently) detect it, and wiring this up wrong can result in strange behaviour and phantom values.