Readable & Writable Streams
Streams come in readable, writable, and duplex types.
Syntax
writable.write(chunk);
writable.end();Streams are Node's tool for handling flowing data. There are four types:
- Readable — you read data from it (a file being read).
- Writable — you write data to it (a file being written).
- Duplex — both readable and writable (a network socket).
- Transform — a duplex stream that changes data as it passes (compression).
All streams are EventEmitters, emitting events like 'data', 'end', and 'error'.
Example
const fs = require('fs');
const out = fs.createWriteStream('numbers.txt');
for (let i = 1; i <= 5; i++) {
out.write(`Line ${i}\n`);
}
out.end();
out.on('finish', () => console.log('All lines written.'));When to use it
- A video-on-demand server uses `fs.createReadStream` to stream large MP4 files to clients without loading them entirely into RAM.
- A data ingestion pipeline uses a Transform stream to decompress a gzipped S3 download on-the-fly before parsing the CSV records.
- A real-time log aggregator uses a Writable stream to forward incoming log chunks to a central logging service as data arrives.
More examples
Four stream types overview
Lists the four stream types in Node's built-in `stream` module and a typical example for each.
const { Readable, Writable, Duplex, Transform } = require('stream');
// Readable -- source of data (e.g. fs.createReadStream)
// Writable -- destination (e.g. fs.createWriteStream)
// Duplex -- both sides (e.g. a TCP socket)
// Transform-- reads+writes+transforms (e.g. zlib.createGzip)Create a custom Readable stream
Creates a Readable stream in object mode that pushes numbers and ends with a `null` sentinel.
const { Readable } = require('stream');
const numberStream = new Readable({
objectMode: true,
read() {},
});
for (let i = 1; i <= 5; i++) numberStream.push(i);
numberStream.push(null); // signal end
numberStream.on('data', n => console.log(n));Transform stream to uppercase
Builds a Transform stream that upper-cases every chunk and wires it between stdin and stdout.
const { Transform } = require('stream');
const upper = new Transform({
transform(chunk, encoding, callback) {
this.push(chunk.toString().toUpperCase());
callback();
},
});
process.stdin.pipe(upper).pipe(process.stdout);
Discussion