Clarify: tee is unsafe for independent consumption since it backpressures to the faster consumer #1233
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Unfortunately, the built-in tee algorithm backpressures to the faster consumer and enqueues chunks without limit to the slower consumer. Therefore, it is generally not safe to use when the two branches are read by independent consumers at different speed, since the consumer will eventually run out of memory.
I believe that this was an unfortunate design mistake because it conflicts with the suggestion in Stream requirements:. You must be able to pipe a stream to more than one writable stream. which said, “The tee stream can use a number of strategies to govern how the speed of its outputs affect the backpressure signals it gives off, but the simplest strategy is to pass aggregate backpressure signals directly up the chain, thus letting the speed of the slowest output determine the speed of the tee.” This also differs from analogous operations in other libraries such as repeated nodejs Readable.pipe or akka-streams Source.alsoTo which backpressure to the slower consumer and only require a bounded queue.
As I said in a node-fetch issue node-fetch/node-fetch#1568, I think this unbounded enqueueing algorithm of
ReadableStream.tee()
from #311 was a mistake. The built-in operations of aReadableStream
should only need a fixed-size queue. , but at this point it is too late to fix it.But at least we can clarify the informational text, which implied that it is safe to
tee()
a stream and consume it independently. It is generally not safe to consume atee()
’d stream independently unless the difference in consumed data between the faster consumer and the slower consumer fits in memory.(See WHATWG Working Mode: Changes for more details.)
Preview | Diff