HTML Syntax, Parsing, and Conformance

Advanced

HTML syntax is not just a visual notation for elements. For text/html, the HTML Standard defines a tokenizer followed by a tree-construction stage. Together they turn a stream of code points into a document tree, including defined recovery behavior for malformed input.

text/html has its own parsing language

The HTML Standard defines rules for resources parsed as text/html. Those rules are distinct from XML parsing. A browser does not need a separate SGML parser to interpret HTML; the HTML parser follows the tokenizer and tree-construction algorithms defined by the Living Standard.

<!-- choose -->?

The pipeline: code points → tokens → tree

The parsing input is a stream of code points. Tokenization recognizes constructs such as start tags, end tags, comments, character data, and the end-of-file token. Each emitted token is then handled by tree construction, which modifies the document tree.

<!-- choose -->?

The tokenizer is a state machine

The tokenizer starts in the data state and moves through states according to the input. States recognize tag openers, tag names, attributes, comments, character references, raw text, and other syntax. Reconsume behavior lets a character be processed again in a newly selected state when the algorithm requires it.

<!-- choose -->?

Character references are parsed, not merely copied

Character references such as & are interpreted by the HTML tokenizer. The source spelling and the resulting character data are therefore different layers of the parsing process.

<p>?</p>

Tree construction has state too

Tree construction is not a simple "append every start tag as a child" algorithm. Its insertion mode and stack of open elements determine how each token is handled. The algorithm can insert elements that were not explicitly written in the source and can close or rearrange structures according to the defined parsing rules.

<body>?

Insertion modes give tree construction context

The tree-construction stage uses insertion modes such as "in body", "in table", "in row", and "in cell". The same token can therefore have different handling depending on where the parser is in the document structure.

<!-- choose -->?

Malformed markup has defined recovery behavior

HTML conformance and HTML parsing are different questions. A source can violate authoring requirements while a conforming user agent still follows the parser algorithm to construct a tree. This defined error handling is a major part of HTML interoperability.

<!-- choose -->?

Raw text and character-reference-aware text states

Some elements cause the tokenizer to use specialized states. Script data and raw text elements are not tokenized exactly like ordinary text. The distinction matters because characters that look like markup in ordinary text can have different significance in these states.

<script>?</script>

Tables have specialized tree construction rules

Table markup is a notable example of context-sensitive tree construction. The parser uses table-specific insertion modes and can use foster parenting when content appears where the table parsing rules do not permit it. This is parsing behavior, not a recommendation to author tables incorrectly.

<!-- choose -->?

Parsing can interact with script execution

The parsing model is reentrant. Script execution can use dynamic markup insertion APIs such as document.write(), which can insert characters into the input being tokenized. This is why the parser model includes concepts such as parser pause behavior and script nesting.

<!-- choose -->?

Conformance is an authoring contract

A conforming document satisfies the authoring requirements of the HTML Standard. A parser, however, is specified to process text/html input including input containing parse errors. Do not use successful parsing as proof that markup is conforming.

<!-- choose -->?

Authoring rule: write for conformance, understand recovery

Use the parser model to understand real documents and debugging results, not as permission to depend on malformed markup. Write according to element content models, required attributes, permitted parents, and syntax rules defined by the HTML Standard. When a browser recovers from an error, treat that as specified interoperability behavior rather than as an authoring pattern.

Normative sources

Primary normative source: WHATWG HTML Living Standard, especially the conformance requirements, common infrastructure, HTML syntax, and parsing sections. No MDN material is used as normative authority.

Normative references: https://html.spec.whatwg.org/multipage/parsing.html and https://html.spec.whatwg.org/multipage/introduction.html#conformance-requirements-for-authors