Buffers & Binary Data
alloc vs allocUnsafe, reading typed values, and avoiding the multibyte-split bug.
A Buffer is a fixed-length view over raw memory outside V8's heap. Two things trip people up: how you allocate and how text maps to bytes.
alloc vs allocUnsafe
Buffer.alloc(n)— zero-filled, safe, slightly slower.Buffer.allocUnsafe(n)— faster, but the memory may contain old data (possibly another request's secrets). Only use it when you immediately overwrite every byte.
const buf = Buffer.alloc(4);
buf.writeUInt32BE(0xDEADBEEF, 0);
console.log(buf); // <Buffer de ad be ef>
console.log(buf.readUInt32BE(0).toString(16)); // deadbeefThe multibyte split bug
A UTF-8 character can span several bytes, and stream chunks split on arbitrary byte boundaries. Calling chunk.toString('utf8') per chunk can slice a character in half and produce mojibake. Use string_decoder.StringDecoder, which buffers partial characters across chunks — or work in lines/records and decode complete units.
Example
// Decoding a UTF-8 stream safely across chunk boundaries, and parsing a
// tiny binary protocol (a length-prefixed message framing).
const { StringDecoder } = require('node:string_decoder');
// 1) Safe incremental UTF-8 decode.
const decoder = new StringDecoder('utf8');
const euro = Buffer.from('€', 'utf8'); // 3 bytes: e2 82 ac
// Simulate the char arriving split across two chunks.
console.log(decoder.write(euro.subarray(0, 2))); // '' (holds partial char)
console.log(decoder.write(euro.subarray(2))); // '€' (completes it)
// 2) Frame messages from a byte stream: [uint32 length][payload]...
function* readFrames(buf) {
let offset = 0;
while (offset + 4 <= buf.length) {
const len = buf.readUInt32BE(offset);
if (offset + 4 + len > buf.length) break; // incomplete frame; wait for more
yield buf.subarray(offset + 4, offset + 4 + len);
offset += 4 + len;
}
}
const msg = (s) => { const b = Buffer.from(s); const h = Buffer.alloc(4); h.writeUInt32BE(b.length); return Buffer.concat([h, b]); };
const wire = Buffer.concat([msg('hello'), msg('world')]);
for (const frame of readFrames(wire)) console.log('frame:', frame.toString());When to use it
- A binary protocol parser reads fixed-size frames from a TCP socket into a Buffer and uses `readUInt16BE` and `readUInt32BE` to extract header fields.
- An image service reads EXIF metadata from a JPEG by slicing a Buffer at known byte offsets without copying the underlying memory.
- A crypto module generates a 256-bit random key with `crypto.randomBytes(32)` and stores it as a Buffer for use in HMAC signing.
More examples
Read and write binary fields
Writes and reads multi-byte integers in big-endian order using Buffer's typed accessor methods.
const buf = Buffer.alloc(8);
buf.writeUInt16BE(1024, 0); // write 2-byte header
buf.writeUInt32BE(99999, 2); // write 4-byte payload length
buf.writeUInt16BE(0xFFFF, 6); // write 2-byte checksum
console.log(buf.readUInt16BE(0)); // 1024
console.log(buf.readUInt32BE(2)); // 99999Slice a Buffer without copy
Uses `subarray` to create a view of a Buffer slice that shares the same underlying memory -- zero copy.
const full = Buffer.from('Hello, World!');
// subarray shares memory with 'full'
const hello = full.subarray(0, 5);
console.log(hello.toString()); // 'Hello'
// slice also shares -- modify slice affects original
hello[0] = 0x68; // lowercase 'h'
console.log(full.toString()); // 'hello, World!'Concatenate multiple Buffers
Uses `Buffer.concat()` to merge an array of Buffer chunks into a single Buffer -- the standard pattern when accumulating stream data.
const chunks = [
Buffer.from('Hello'),
Buffer.from(', '),
Buffer.from('World!'),
];
const combined = Buffer.concat(chunks);
console.log(combined.toString()); // 'Hello, World!'
console.log(combined.length); // 13
Discussion