Node.js streams and backpressure: when the writer is slower than the reader Intermediate

The problem

An API receives file uploads and writes each chunk to object storage. The client sends faster than storage writes. If the code keeps calling write() without reading the return value, chunks pile up in the process’s memory.

A small file still passes tests; 200 concurrent 500 MB uploads make RSS grow, GC run constantly, latency rise, and finally OOM. Adding RAM only delays the failure.

Backpressure is the consumer’s signal saying “do not push faster than I can process”. For a Writable, that signal is write() returning false plus the drain event. This article uses a deliberately slow Writable to make the signal visible, then shows how to write correctly.

Signal table

One table for every Writable signal:

Signal	Means	Does not mean
write() returns true	After accepting the chunk, the buffer is still below highWaterMark	The chunk has been durably written
write() returns false	The chunk was accepted; do not write the next chunk until drain	The chunk was rejected and must be rewritten
writableNeedDrain	true when the buffer is full and the stream will emit drain	A memory metric
drain	The buffer has been fully consumed; you can write again	The whole pipeline is done
finish	After end(), all data has been flushed to the underlying system	Remote storage has committed
close	The stream and its underlying resource (for example a file descriptor) are closed; no more events follow	The write succeeded (it is also emitted on destroy)
error	An error occurred; the stream is usually destroyed right after

highWaterMark is the threshold where the signal flips, not a hard memory limit: if you keep calling write() despite false, Node keeps buffering.

Default highWaterMark

Stream type	Default	Notes
Byte streams (Writable, Readable, Transform)	64 KiB since Node 22	16 KiB before Node 22
Object mode	16 objects	Counted in objects, not bytes
fs.createReadStream / createWriteStream	64 KiB	Measured on Node 24.17

Measured on Node 24.17.0: new Writable().writableHighWaterMark is 65536, object mode is 16, stream.getDefaultHighWaterMark(false) is 65536 and (true) is 16. Node 22.0.0 lists “stream: bump default highWaterMark” among its semver-major changes.

If the service runs on several Node versions, do not rely on the default: set highWaterMark explicitly or read stream.getDefaultHighWaterMark().

Buffer fill over time

The demo writes 8-byte chunks into a Writable with highWaterMark: 16 and a sink that takes 15 ms per chunk. Actual trace (Node 24.17):

Time	Event	writableLength	writableNeedDrain
+0 ms	write #1 returns true	8 / 16	false
+0 ms	write #2 returns false	16 / 16	true
+32 ms	drain, the producer resumes	0	false
+32 ms	write #3 returns true, write #4 returns false	16 / 16	true
+64 ms	drain	0	false

Each cycle the producer pauses about 32 ms, exactly the time the sink needs for two chunks. That is backpressure: the producer’s speed is pulled down to the consumer’s. The full 10-chunk demo prints { chunks: 10, paused: 5 }.

The producer pauses when the Writable buffer reaches the threshold and resumes after drain

Enlarge diagram

Producer → Writable bufferchunk

◀ Prev▶ PlayNext ▶↺ RestartStep 1/4

The producer calls write. The chunk lands in the Writable's buffer and is gradually passed on to the sink.

Writing correctly

Option 1: use pipeline (the default choice). stream/promises.pipeline wires up backpressure, propagates errors and cleans up resources for the whole chain:

import { pipeline } from 'node:stream/promises';

const ac = new AbortController();
res.on('close', () => { if (!res.writableFinished) ac.abort(); }); // client disconnected

await pipeline(req, gzip, storageWritable, { signal: ac.signal });

When the signal is aborted, pipeline calls destroy with an AbortError on the whole chain (per the Node docs). If one stream is destroyed midway, pipeline rejects with ERR_STREAM_PREMATURE_CLOSE (verified on Node 24.17).

Option 2: write manually in a loop. Needed when the source is not a stream.

The commonly missed point: await once(sink, 'drain') rejects automatically if the sink emits error while waiting, but if the sink is destroy()ed without an error, neither drain nor error is ever emitted and the loop hangs forever (measured: still hanging after 300 ms).

You must also wait on close:

import { once } from 'node:events';

async function writeAll(sink: Writable, chunks: Iterable<Buffer>) {
  for (const chunk of chunks) {
    if (sink.write(chunk)) continue;
    const ac = new AbortController();
    try {
      await Promise.race([
        once(sink, 'drain', { signal: ac.signal }),
        once(sink, 'close', { signal: ac.signal }).then(() => {
          throw new Error('sink closed early');
        }),
      ]);
    } finally {
      ac.abort(); // remove the remaining listener
    }
  }
  sink.end();
  await once(sink, 'finish');
}

With a sink destroyed midway, this function throws sink closed early; with a normal sink it completes. Both were verified on Node 24.17.

Real-world example

500 MB uploads make the API OOM

The handler collects every chunk with Buffer.concat before sending to storage. With 40 concurrent 500 MB uploads, the process needs about 20 GB just to hold the bodies (40 times 500 MB).

The team switches to pipeline(req, gzip, storageWritable, { signal }), limits concurrent uploads to 10 and caps body size, and sets a timeout for storage (timeout budget).

After the change, memory per upload is roughly the sum of the chain’s buffers (a few times 64 KiB plus the compression library’s and SDK’s buffers), times 10.

The final number still has to be measured with a heap profile, because gzip’s and the SDK’s internal buffers are not controlled by your highWaterMark.

Two points about the reading side: for await (const chunk of readable) followed by writable.write(chunk) still has to handle false or use pipeline; a read loop does not yield to an arbitrary Writable by itself.

With res.write() in HTTP, a slow-reading client produces false in exactly the same way, and a route generating unbounded data without waiting for drain grows memory from a single client.

Production checklist

1. Use stream/promises.pipeline for every stream chain; write manually only when the source is not a stream.
2. Pass an AbortSignal into pipeline and abort when the client disconnects or the deadline expires.
3. When writing manually, handle false with drain, and also listen for error and close so you never hang forever.
4. Set highWaterMark explicitly if behavior must not depend on the Node version (the default changed in Node 22).
5. Limit concurrent uploads/streams and body size: per-stream backpressure does not replace an overall budget.
6. Monitor writableLength (or writableNeedDrain) of important streams and the process’s RSS.
7. Treat data as durable only after storage confirms it, not after finish of a hand-written adapter.

Common mistakes

Common pitfall

Treating write() returning false as a failed write and rewriting the chunk. The chunk is already in the buffer; rewriting it duplicates data. false only means wait for drain before the next chunk.

Common pitfall

Ignoring false and trusting the stream to cap memory. Node keeps buffering. With a fast producer or unbounded data, the process still runs out of memory.

Common pitfall

Waiting for finish and then reporting “saved”. finish only says the data has been flushed to the stream’s underlying layer. With a hand-written Writable that calls its callback early, remote storage may not have committed yet.

Summary

Key takeaways

• write() === false means wait for drain before the next chunk; the chunk just written is not dropped.
• The default highWaterMark is 64 KiB (bytes) and 16 (object mode) since Node 22; it is a signal threshold, not a memory cap.
• finish means flushing completed after end(); close means the resource is closed (also on error or destroy).
• pipeline with an AbortSignal is the right default; when writing manually, listen for both error and close.
• Per-stream backpressure does not replace limits on concurrency and total size.

Self-check: (1) After write() returns false, do you rewrite that chunk? (2) When does await once(sink, 'drain') hang forever? (3) How do finish and close differ when the stream is destroy()ed?

Run the demo:

npx --prefix examples ts-node examples/src/node-backpressure/backpressure-demo.ts

References

• Stream API — Node.js
• Backpressuring in Streams — Node.js
• Node.js 22.0.0 release notes
• events.once — Node.js