JavaScript Unicode-aware Strings

Advanced

JavaScript strings are sequences of 16-bit values. That sounds like a small implementation detail until text contains characters outside the Basic Multilingual Plane, combining marks, or malformed surrogate code units. This lesson builds a precise mental model so you can choose string operations based on what you actually mean by a “character”.

The three boundaries to keep separate are **code units**, **code points**, and **grapheme clusters**. ECMAScript string indexing and `length` use UTF-16 code units. Code-point-aware iteration can combine a valid surrogate pair into one code point. A grapheme cluster is closer to what a reader perceives as one text unit, and locale-sensitive segmentation belongs to `Intl.Segmenter`, which is covered more fully in the internationalization unit.

Code units: the indexing model

A JavaScript String has a length equal to the number of 16-bit code units. Basic characters commonly occupy one code unit, while many supplementary Unicode code points occupy two code units as a surrogate pair. That is why `length` and bracket indexing are not reliable measures of user-perceived characters.

See the code-unit length

?

Code points and surrogate pairs

A Unicode code point is a numeric value in the Unicode range. A supplementary code point is represented in a JavaScript string by two UTF-16 code units: a leading surrogate followed by a trailing surrogate. A valid pair is interpreted as one code point by code-point-aware operations.

Read a code point with codePointAt

?

Create text from code points

Use `String.fromCodePoint()` when the input you have is a Unicode code point value. It differs from `String.fromCharCode()`, which works with UTF-16 code units and therefore requires more care for supplementary characters.

Build a supplementary character from its code point

?

Iterate by code point

A String iterator is code-point-aware. `for...of`, spread syntax, and `Array.from()` use that iterator, so a valid surrogate pair is yielded as one string element instead of two separate code units.

Compare indexing with iteration

?

Code points are not grapheme clusters

Even code-point iteration is not the same as counting what a person sees as one character. A visible unit can contain multiple code points, such as a base letter plus a combining mark, or an emoji sequence joined from several code points. Do not promise users that `[...text].length` is a “character count”.

One visible unit can contain multiple code points

?

Well-formed and ill-formed UTF-16

ECMAScript strings can contain an unpaired surrogate. Such a string is not a well-formed UTF-16 sequence, even though it is still a valid JavaScript String value. Modern String APIs expose explicit checks and repair operations so code can decide what to do at boundaries.

Detect and repair an unpaired surrogate

?

Normalization is a separate problem

Two strings can represent canonically equivalent text while containing different code-point sequences. `normalize()` explicitly requests Unicode normalization; it does not silently happen for ordinary string operations. Choose a normalization form deliberately when your application needs canonical or compatibility equivalence.

Compare canonically equivalent spellings

?

Indexing, slicing, and boundaries

Methods such as `slice()`, `substring()`, and bracket indexing use string indices measured in code units. If you cut through the middle of a surrogate pair, you can create an unpaired surrogate. Code-point-aware traversal is safer when the boundary is intended to be between code points.

A code-unit slice can split a surrogate pair

?

Escapes describe code units or code points depending on syntax

JavaScript source has several Unicode-related escape forms. `\uXXXX` describes one UTF-16 code unit, while `\u{...}` in Unicode-aware syntax can express a full code point value. Escape syntax does not change the String storage model.

Compare the two Unicode escape styles

?

Case conversion is not Unicode identity

Case conversion can depend on language-sensitive rules, and changing case is not the same operation as normalization or case-insensitive identity. The basic String methods provide default case conversion; locale-sensitive variants belong to the internationalization layer.

Case conversion can change length

?

Searching is still code-unit based

Many familiar string operations return positions in the same code-unit index space used by `length`. This is useful and consistent, but it means a returned index is not automatically a code-point count.

Search positions are code-unit indices

?

Choose the boundary your feature actually needs

Use code-unit operations when you intentionally work with ECMAScript string indices. Use code-point iteration when you need to avoid splitting valid surrogate pairs. If you need user-perceived text elements, use a segmentation solution such as `Intl.Segmenter` rather than inventing a code-point count as a substitute.

Pick a representation for a simple text inspector

?

`charCodeAt()` and `codePointAt()` answer different questions

`charCodeAt()` returns the numeric value of one UTF-16 code unit. `codePointAt()` can interpret a valid surrogate pair as one code point. Choose between them based on whether your algorithm is deliberately operating on code units or Unicode code points.

Compare code-unit and code-point reads

?

Well-formedness and normalization are independent

A string can be well-formed UTF-16 and still use a different normalization form from another canonically equivalent string. Conversely, repairing an unpaired surrogate does not normalize the rest of the text. Treat these as separate policies.

Check the two concerns separately

?

Authoring rules for Unicode text

Keep these rules close to your code: never equate `length` with visible-character count; do not index through an emoji sequence assuming one code unit per character; use `for...of` or another code-point-aware approach when surrogate pairs matter; use `normalize()` only with an explicit equivalence requirement; validate or repair untrusted text when well-formed UTF-16 matters; and treat grapheme segmentation as a separate problem.

A compact Unicode-safe iteration pattern

?

What the ECMAScript specification actually defines

ECMA-262 defines String values as sequences of 16-bit elements and specifies how operations such as `codePointAt`, the String iterator, normalization, and well-formedness checks interpret those values. It does not redefine every user-perceived text boundary as a “character”. Keeping the language model and the user-interface text model separate prevents subtle bugs.

Inspect all three levels explicitly

?