util.TextDecoder
Decodes bytes in a given character encoding into a string, buffering partial characters between chunks.
new TextDecoder([encoding[, options]])- Added in Node.js v8.3.0
Parameters
| Parameter | Type | Description |
|---|---|---|
$encodingoptional | string= 'utf-8' | Identifies the encoding that this TextDecoder instance supports. Which labels are accepted depends on the ICU data the Node.js binary was built with; 'utf-8', 'utf-16le' and 'utf-16be' are always available. |
$optionsoptional | Object | Configuration for error handling and byte order mark treatment. |
$options.fataloptional | boolean= false | true if decoding failures are fatal. When false, malformed bytes become U+FFFD replacement characters instead of throwing. |
$options.ignoreBOMoptional | boolean= false | When true, a leading byte order mark is not treated as a BOM and is passed through as a U+FEFF character. When false, a leading BOM is consumed and removed from the output. |
$inputoptional | ArrayBuffer | DataView | TypedArray= undefined | Argument to textDecoder.decode(): an ArrayBuffer, DataView, or TypedArray instance containing the encoded data. |
$options.streamoptional | boolean= false | Argument to textDecoder.decode(): true if additional chunks of data are expected. Incomplete byte sequences at the end of the input are buffered internally and emitted after the next call. |
Return value
TextDecoderA TextDecoder instance bound to the given encoding, with a decode() method that turns byte input into a string.
Errors
The constructor throws a RangeError with code ERR_ENCODING_NOT_SUPPORTED for an encoding label the build does not support, and decode() throws a TypeError on malformed input when fatal is true.
Changelog
- PHP v11.0.0TextDecoder is exposed on the global object, so require('node:util') is no longer needed to reach it.
TextDecoder turns raw bytes into a JavaScript string using a named character encoding. It is the WHATWG API, identical to the one in browsers, and is available as a global as well as from node:util.
When to reach for it
Two cases justify it over buf.toString(). The first is any encoding Node's own Buffer does not speak — windows-1252, shift_jis, gbk, iso-8859-7. The second, and the more common bug, is decoding a stream. A UTF-8 character can be up to four bytes, and a chunk boundary lands wherever the read happened to stop. Calling toString() on each chunk splits that character in half and produces � garbage that no amount of concatenation repairs.
Passing { stream: true } makes the decoder hold back trailing incomplete bytes and prepend them to the next call, so the character is reassembled correctly across the seam.
What trips people up
- You must make a final call with no arguments (or
stream: false) to flush the buffer. Skip it and a trailing truncated character is silently dropped. - A decoder is stateful. Reuse one instance for one stream, and create a fresh one for the next.
- Availability depends on the build. A
small-icubinary supports far fewer labels, and an ICU-less build supports onlyutf-8,utf-16le,utf-16beandibm866.
Example
When to use it
- Decode a large UTF-8 file read through createReadStream without corrupting characters at chunk boundaries.
- Read a legacy CSV exported as windows-1252 or shift_jis into correct JavaScript strings.
- Reject corrupted uploads early by decoding with fatal: true instead of accepting replacement characters.
- Strip the byte order mark Excel writes at the top of a UTF-8 CSV before parsing it.
Frequently asked questions
Why do I get question marks or diamond characters when decoding a buffer?
Those are U+FFFD replacement characters, produced when a multi-byte character was cut in half — usually because each stream chunk was decoded separately with buf.toString(). Use one TextDecoder instance and pass { stream: true } on every chunk so partial bytes carry over, then call decode() with no arguments at the end to flush. If the bytes are genuinely not UTF-8, pick the right encoding label instead, or construct with fatal: true to be told about it rather than shown a placeholder.
What is the difference between TextDecoder and buffer.toString()?
buffer.toString() supports only Node's own encoding list — utf8, utf16le, latin1, ascii, base64, hex — and has no memory between calls, while TextDecoder supports the full WHATWG label set (windows-1252, shift_jis, gbk, big5 and more, subject to the ICU data in your build) and is stateful, so it can carry an incomplete character across chunk boundaries. Reach for toString() on a complete buffer of UTF-8; reach for TextDecoder for streams or legacy encodings.
Do I need to require node:util to use TextDecoder in Node.js?
No. Since Node.js v11.0.0 TextDecoder is a global, so you can call new TextDecoder() directly with no import at all. Importing { TextDecoder } from 'node:util' gives you the exact same class and is mainly useful for making the dependency explicit in code that is also linted for browser or isomorphic use.
More examples
Decoding a file stream chunk by chunk
Every Japanese character here is three bytes, so a 7-byte read window never lands on a character boundary. The output shows the text reassembled intact: the decoder carried the leftover 1 or 2 bytes from each chunk into the next call. Replacing decode(chunk, { stream: true }) with chunk.toString() would produce replacement characters at every one of those seams.
import fs from 'node:fs';
import { TextDecoder } from 'node:util';
fs.writeFileSync('poem.txt', 'ある日、森の中\nくまさんに出会った\n');
(async () => {
const decoder = new TextDecoder('utf-8');
// A tiny highWaterMark guarantees chunk boundaries fall inside characters.
const stream = fs.createReadStream('poem.txt', { highWaterMark: 7 });
let text = '';
for await (const chunk of stream) {
text += decoder.decode(chunk, { stream: true });
}
text += decoder.decode(); // flush whatever bytes are still held back
console.log(JSON.stringify(text));
console.log('lines:', text.trimEnd().split('\n').length);
fs.unlinkSync('poem.txt');
})();
// Output:
// "ある日、森の中\nくまさんに出会った\n"
// lines: 2fatal: true to reject malformed bytes
The default decoder silently substitutes U+FFFD, so corrupted input flows on into your database as a visible replacement character and is discovered weeks later. With fatal: true the same bytes throw a TypeError you can catch at the ingest boundary. The third case shows that an encoding label the build cannot support fails immediately in the constructor with a RangeError carrying code ERR_ENCODING_NOT_SUPPORTED — check it at startup rather than per request.
const { TextDecoder } = require('node:util');
// 0xFF is never valid in UTF-8.
const bad = new Uint8Array([0x48, 0x69, 0xff]);
const lenient = new TextDecoder('utf-8');
console.log('lenient:', JSON.stringify(lenient.decode(bad)));
const strict = new TextDecoder('utf-8', { fatal: true });
try {
strict.decode(bad);
} catch (err) {
console.log('strict:', err.name + ':', err.message);
}
// Unsupported labels fail at construction, not at decode time.
try {
new TextDecoder('bogus-8');
} catch (err) {
console.log('bad label:', err.code, err.name);
}
// Output:
// lenient: "Hi�"
// strict: TypeError: The encoded data was not valid for encoding utf-8
// bad label: ERR_ENCODING_NOT_SUPPORTED RangeErrorByte order marks and legacy encodings
The first pair shows the inverted option: the default (ignoreBOM: false) removes the BOM, while ignoreBOM: true leaves U+FEFF in place as a real character — which is why a header parsed from such a file compares unequal to "id" despite looking identical. The second pair shows the main reason to use TextDecoder at all for non-streaming work: byte 0xE9 is invalid UTF-8 but is é in windows-1252, an encoding Buffer.toString() cannot handle.
const { TextDecoder } = require('node:util');
// A UTF-8 CSV as exported by Excel: EF BB BF then "hi".
const withBom = Buffer.from([0xEF, 0xBB, 0xBF, 0x68, 0x69]);
const stripped = new TextDecoder('utf-8').decode(withBom);
const kept = new TextDecoder('utf-8', { ignoreBOM: true }).decode(withBom);
console.log('default :', JSON.stringify(stripped), 'length', stripped.length);
console.log('ignoreBOM :', JSON.stringify(kept), 'length', kept.length);
console.log('first code point kept:', kept.codePointAt(0).toString(16));
// Buffer has no windows-1252; TextDecoder does.
const latin = Buffer.from([0x63, 0x61, 0x66, 0xE9]);
console.log('as utf-8 :', JSON.stringify(latin.toString('utf8')));
console.log('as windows-1252:', new TextDecoder('windows-1252').decode(latin));
// Output:
// default : "hi" length 2
// ignoreBOM : "hi" length 3
// first code point kept: feff
// as utf-8 : "caf�"
// as windows-1252: café
Discussion