Encoding
TextEncoder converts strings into UTF-8 bytes; TextDecoder converts
UTF-8 bytes back into strings. Use them whenever you move between text
and raw bytes: file contents, request bodies, encoded data.
TextEncoderStream and TextDecoderStream do the same conversion as a
TransformStream, so you can pipe data through them chunk by chunk.
new TextEncoder()creates an encoder; every encoder produces UTF-8.encoder.encode(string)returns aUint8Arrayof UTF-8 bytes.encoder.encodeInto(source, destination)encodes into an existingUint8Arrayand returns{ read, written }counts.new TextDecoder(label, options)creates a decoder. Thelabelmust identify UTF-8 ("utf-8","utf8", or"unicode-1-1-utf-8"); it defaults to"utf-8". Thefataloption makes decoding throw on malformed input instead of substituting replacement characters.decoder.decode(input)returns the decoded string.decoder.encodinganddecoder.fatalreflect the constructor options.TextDecoderdoes not supportignoreBOM; useTextDecoderStreamfor BOM control.new TextEncoderStream()turns a stream of strings into a stream ofUint8Arraychunks.new TextDecoderStream(label, options)turns a stream of bytes into a stream of strings. It keeps incomplete sequences between chunks.- Both streams expose
readableandwritable, so they work withpipeThrough().
Usage#
const encoder = new TextEncoder();
const bytes = encoder.encode("Hello, world!");
console.log(bytes.byteLength); // 13
await fs.writeFile("/out.json", encoder.encode(JSON.stringify({ ok: true })));
Encoding into an existing buffer (useful for fixed-size buffers):
const buffer = new Uint8Array(64);
const { read, written } = new TextEncoder().encodeInto("geartrain", buffer);
console.log(read, written); // 7 7
Decoding bytes back into text:
const decoder = new TextDecoder();
const bytes = await fs.readFile("/data.json");
const data = JSON.parse(decoder.decode(bytes));
Strict decoding:
const strict = new TextDecoder("utf-8", { fatal: true });
try {
strict.decode(maybeCorrupt);
} catch (err) {
console.log("input was not valid UTF-8");
}
Encoding a stream of strings:
const text = new ReadableStream({
start(controller) {
controller.enqueue("Hello, ");
controller.enqueue("world!");
controller.close();
},
});
const encoded = [];
await text
.pipeThrough(new TextEncoderStream())
.pipeTo(
new WritableStream({
write(chunk) {
encoded.push(chunk);
},
}),
);
console.log(encoded.length); // 2
console.log(new TextDecoder().decode(encoded[0])); // Hello,
Decoding a stream of bytes, with a character split across chunks:
const bytes = new ReadableStream({
start(controller) {
controller.enqueue(new Uint8Array([0x48, 0x69]));
controller.enqueue(new Uint8Array([0xe2]));
controller.enqueue(new Uint8Array([0x82, 0xac]));
controller.close();
},
});
let text = "";
await bytes
.pipeThrough(new TextDecoderStream())
.pipeTo(
new WritableStream({
write(chunk) {
text += chunk;
},
}),
);
console.log(text); // Hi€
Strict decoding works on the stream too. A malformed byte errors both
sides with a TypeError:
const strict = new TextDecoderStream("utf-8", { fatal: true });
Differences from the standard#
- Only UTF-8 is supported. Any other encoding label throws a
RangeErroronnew TextDecoder(label)ornew TextDecoderStream(label), while the standard defines a large label registry for legacy encodings. TextDecoder.decode()does not support thestreamoption and throws if one is passed. UseTextDecoderStreamwhen a multi-byte character may be split across chunks.TextDecoderdoes not implement BOM control: itsignoreBOMis always false anddecode()keeps a leading byte order mark.TextDecoderStreamfollows the standard and removes one leading BOM unlessignoreBOMis true.TextEncoder.encode()does not support the deprecatedstreamoption; passing one throws aTypeError.- Passing
fataltodecode()throws aTypeError; the standard ignores unknown options ondecode()(use the constructor instead).