stream.Transform
Create a duplex stream that rewrites each chunk on its way through.
new stream.Transform([options])- Added in Node.js v0.9.4
Parameters
| Parameter | Type | Description |
|---|---|---|
$optionsoptional | Object | Passed to both the Writable and Readable constructors, so every option of either is accepted, plus the two below. |
$options.transformoptional | Function | Implementation for the stream._transform() method, called as transform(chunk, encoding, callback). |
$options.flushoptional | Function | Implementation for the stream._flush() method, called as flush(callback) before the stream closes so trailing data can be emitted. |
$options.objectModeoptional | boolean= false | Whether both sides of the stream operate in object mode. Use readableObjectMode or writableObjectMode to set only one side. |
$options.decodeStringsoptional | boolean= true | Whether strings written in are converted to Buffers before reaching _transform(). |
$options.highWaterMarkoptional | number= See stream.getDefaultHighWaterMark() | Buffer level at which writes into the transform start returning false. |
Return value
stream.TransformA new transform stream instance, which is both readable and writable.
Errors
Writing to a Transform that provides no _transform() implementation throws ERR_METHOD_NOT_IMPLEMENTED.
Changelog
- PHP v22.0.0The default highWaterMark was raised, changing how much data a transform buffers before applying backpressure.
- PHP v14.0.0autoDestroy now defaults to true, so a transform destroys itself and emits 'close' after 'end'.
A Transform is a duplex stream whose readable side is produced from its writable side. You implement _transform(chunk, encoding, callback); call callback(null, output) to emit one chunk, this.push(x) repeatedly to emit several, or callback() with nothing to drop the chunk entirely. That last form is how you write a filter.
When to reach for it
Whenever a middle stage in a pipeline needs to change the data: compressing, parsing NDJSON into objects, redacting secrets from a log stream, re-encoding text. zlib.createGzip() and crypto.createCipheriv() are Transforms, which is why they slot into pipeline() without adaptation.
What trips people up
A Transform is paused until something reads from it. Write into one that has not been piped anywhere and has no 'data' handler, and it silently fills its buffer and appears to hang. In practice, always put a Transform inside a pipeline() rather than driving it by hand.
Chunk boundaries are the other trap. Nothing guarantees a chunk is a whole line or a whole record, so a line-oriented transform must keep a remainder in instance state and emit it from _flush(). Skipping _flush() quietly loses the last line of every file that does not end in a newline.
Use readableObjectMode when a transform turns bytes into objects.
Example
When to use it
- Parse newline-delimited JSON into objects in the middle of a pipeline
- Redact tokens or emails from a log stream before it is written to disk
- Convert CSV rows into records for a database loader
- Count or checksum bytes while they pass through, without buffering the whole file
Frequently asked questions
How do I modify data as it passes through a stream in Node.js?
Use a stream.Transform and implement _transform(chunk, encoding, callback), emitting the rewritten chunk with callback(null, output). Place the transform between source and destination inside pipeline().
How do I filter out chunks in a transform stream?
Call callback() with no data argument for chunks you want to drop; only chunks passed to callback(null, value) or this.push(value) reach the readable side.
Why does my transform stream lose the last line?
Because the final partial chunk is still held in your buffer when the writable side ends, and _flush() was not implemented to emit it. Implement _flush(callback) and push the remainder there before calling callback().
More examples
Splitting a byte stream into lines
The source deliberately splits "grace" across two chunks. Buffering the remainder reassembles it, so the output is line: ada, line: grace, line: alan, done, err: null. Remove _flush() and "alan" — the trailing line with no newline after it — silently disappears.
import { Transform, pipeline, Readable } from 'node:stream';
class LineSplitter extends Transform {
constructor() {
// Bytes in, objects (strings) out.
super({ readableObjectMode: true });
this.remainder = '';
}
_transform(chunk, encoding, callback) {
const parts = (this.remainder + chunk).split('\n');
// The last element may be a partial line; hold it back.
this.remainder = parts.pop();
for (const line of parts) this.push(line);
callback();
}
_flush(callback) {
// Without this, a file not ending in a newline loses its last line.
if (this.remainder) this.push(this.remainder);
callback();
}
}
pipeline(
Readable.from(['ada\ngra', 'ce\nalan']), // note the split mid-word
new LineSplitter(),
async function (lines) {
for await (const line of lines) console.log('line:', line);
},
(err) => console.log('done, err:', err ?? null),
);Filtering and gzipping in one pipeline
Shows two things at once: calling callback() with no second argument discards the chunk, which is how filters are written, and zlib's compressors are ordinary Transform streams that drop straight into the same chain. Prints "compressed" on success.
const { pipeline, Transform } = require('node:stream');
const { createGzip } = require('node:zlib');
const fs = require('node:fs');
// Dropping a chunk is just calling callback() with no data.
const dropComments = new Transform({
decodeStrings: false,
transform(line, encoding, callback) {
if (String(line).startsWith('#')) return callback();
callback(null, line);
},
});
pipeline(
fs.createReadStream('config.txt', 'utf8'),
dropComments,
createGzip(), // zlib streams are Transforms too
fs.createWriteStream('config.txt.gz'),
(err) => console.log(err ? `failed: ${err.message}` : 'compressed'),
);
Discussion