Writable Streams & Backpressure
Why write() returns a boolean, the 'drain' event, and cork/uncork batching.
writable.write(chunk) returns a boolean, and ignoring it is the number-one cause of runaway memory in Node. false means the internal buffer has passed highWaterMark and you should stop writing until the 'drain' event fires. Keep writing anyway and the buffer grows without bound.
const fs = require('node:fs');
const out = fs.createWriteStream('big.txt');
// Correct: honor the return value.
if (!out.write('some data')) {
out.once('drain', () => out.write('more data'));
}Producing faster than you can write
When a fast producer feeds a slow writable in a loop, you must pause on a false return and resume on 'drain'. This is exactly the bookkeeping that pipe/pipeline do for you — which is why you should prefer them. When you must write by hand, do it right.
cork / uncork
cork() buffers many small writes and flushes them as one on uncork(), reducing syscalls for chatty producers. Pair it with process.nextTick(() => stream.uncork()) to batch a burst automatically.
Example
// Manually writing from a fast producer to a slow writable, WITH
// backpressure. Returns a promise that resolves when everything is flushed.
const { once } = require('node:events');
async function writeAll(writable, produce) {
for (let i = 0; i < 1_000_000; i++) {
const chunk = produce(i);
// write() === false means the buffer is full; wait for 'drain'.
if (!writable.write(chunk)) {
await once(writable, 'drain');
}
}
writable.end();
await once(writable, 'finish');
}
// Demonstration with a deliberately slow writable.
const { Writable } = require('node:stream');
const slow = new Writable({
highWaterMark: 16 * 1024,
write(chunk, _enc, cb) { setTimeout(cb, 1); }, // simulate slow sink
});
await writeAll(slow, (i) => `line ${i}\n`);
console.log('all lines written with bounded memory');When to use it
- A logging library implements a custom Writable stream that batches log lines and flushes them to an external log service every 100 lines or 1 second.
- An upload handler pipes an incoming multipart stream to a custom Writable that saves each file part to object storage without loading it into RAM.
- A data exporter writes millions of JSON records one at a time to a Writable file stream, letting backpressure from the OS buffer the writes efficiently.
More examples
Create a custom Writable stream
Extends `Writable` with a `_write` method that upper-cases each chunk before writing it to stdout.
const { Writable } = require('stream');
class UpperCaseWriter extends Writable {
_write(chunk, encoding, callback) {
process.stdout.write(chunk.toString().toUpperCase());
callback(); // signal that this chunk is done
}
}
const writer = new UpperCaseWriter();
writer.write('hello ');
writer.write('world');
writer.end('\n');Check and respond to backpressure
Checks the boolean returned by `.write()` and waits for `'drain'` before writing more -- correct backpressure handling.
const { createWriteStream } = require('fs');
const dest = createWriteStream('output.txt');
function write(data) {
const ok = dest.write(data);
if (!ok) {
// Writable buffer full -- stop producing
dest.once('drain', () => write(nextChunk()));
}
}Writable stream finish event
Listens to the `finish` event (all data flushed) and `error` event on a file write stream.
const { createWriteStream } = require('fs');
const ws = createWriteStream('log.txt');
ws.write('Line 1\n');
ws.write('Line 2\n');
ws.end('Line 3\n');
ws.on('finish', () => console.log('All data flushed to disk'));
ws.on('error', err => console.error('Write error:', err));
Discussion