11.4 Shadow Traffic: mirror

Advanced

What you will learn

1. What mirror does — and the three guarantees that make it safe.

2. The config: mirror location, $request_uri in the target, body duplication.

3. Proving it worked: the sink's log vs the client's response (real captures).

4. The production plays: dark launches, replay for load tests, migration shadowing — and the traps (writes! costs! PII!).

1. Three guarantees

mirror /_mirror; in a location duplicates the request: NGINX fires a background subrequest with the same method, URI, and (via mirror's mirror_request_body on, the default) the same body at the location you name, while the original request proceeds untouched. The contract has exactly three clauses:

1. The client cannot feel it. The mirror runs beside the main content handler, never in front of it. The client's response — status, headers, timing — comes solely from the main path.

2. The mirror's response is discarded. Bytes, status, headers — thrown away. The mirror is a question you ask without waiting for (or caring about) the answer.

3. Mirror failures are invisible. If the sink is slow, down, or on fire, the main request continues. (Caveat: a mirror that hangs keeps worker resources until proxy_timeout — mirror to something fast, or set a short proxy_connect_timeout in the mirror location.)

That trio is what makes dark launches possible: run v2 as the mirror of v1's traffic, diff the sink's behavior against reality, and promote with evidence instead of hope.

2. The config

The :9303 server, complete:

server {
    listen      9303;
    server_name mirror.example.test;

    location / {
        mirror  /_mirror;                    # fire the shadow copy
        proxy_pass  http://127.0.0.1:9002;   # the REAL backend (b-node-2)
    }

    location = /_mirror {
        internal;                            # shadow door: clients 404
        # $request_uri preserves the client's ORIGINAL path+args even
        # though this location's own URI is /_mirror — that's the trick
        # that makes the copy faithful:
        proxy_pass  http://127.0.0.1:9090/mirror$request_uri;
    }
}

Two details carry the pattern:

• /mirror$request_uri — the sink receives /mirror/orders/42?flag=1 for a client request to /orders/42?flag=1. The fixed prefix is this sink's convention (it logs the path it received); the $request_uri part is the faithful original, args included. Without $request_uri, every mirrored request would arrive as /_mirror — a useless shadow.

• The mirror location is a normal location — you can give it its own proxy_set_header (inject X-Shadow: true so the sink knows), its own timeouts, even proxy_cache (replay without hammering the shadow target).

3. The receipts

One POST through :9303, then both sides of the story (real captures):

What the client saw — the main backend's echo, nothing else:

main:  "server": "b-node-2",
main:  "path": "/orders/42?flag=1",

What the sink logged — same request, one line of JSON (the sink, b-mirror, appends to its log and answers 202 to nobody):

{"ts":"2026-09-23T01:07:37.932Z","method":"POST","path":"/mirror/orders/42?flag=1","body":"shadow-payload-123"}

Read them together: same method ( POST), same path ( /orders/42? flag=1 inside the sink's /mirror prefix), same body ( shadow-payload-123 — mirror_request_body on at work), and the client's response contained none of the sink's answer ( mirrored appears nowhere in what the client received — the capture above shows exactly that).

4. The production plays

The traps — this list is the lesson's fine print:

1. Never mirror non-idempotent writes blindly. POST /payments mirrored = double charge. Mirror reads always; mirror writes only when the sink is explicitly designed for it (dedup keys, idempotency tokens) or when the "sink" only records (logs the request instead of executing it).

2. Mirror traffic is real traffic — it costs the sink CPU, bandwidth, and money if the sink is a paid API. rate-limit inside the mirror location, or sample (a map on a random variable → mirror only 10% — Lesson 7.4's split_clients is the sampler you want).

3. PII multiplies. Every header/body you shadow is another copy of secrets to govern. Strip what the sink doesn't need ( proxy_set_header Authorization "" in the mirror location).

4. Ordering is not guaranteed — the shadow may lag the main path. Diff-tolerant consumers only.

5. Bodies buffer in memory while duplicated — a 2 GB upload with mirror_request_body on is read in full first (memory, or a temp file past client_body_buffer_size) and transmitted twice. For giant payloads, turn mirror_request_body off and shadow the metadata only.

Try it yourself

• curl -si -X POST -d shadow-payload-123 http://127.0.0.1:9303/orders/42?flag=1 — then check all four facts: the client saw the main backend, the client never saw the mirror's answer, the sink logged the request, the body was duplicated (the receipts above show each).

• Feel the invisibility: kill the mirror sink, hit :9303 — 200s keep flowing (watch the error log grumble about the mirror connect failure), then revive.

• Add mirror /_mirror2; with a second target — mirrors compose; fan-out is one line each.

Q&A

Q: Does the mirror add latency to the client? A: The handoff itself is non-blocking — the main path doesn't wait. Second-order costs exist (worker cycles, bandwidth), which is why you sample or rate-limit the mirror, not the main path.

Q: Why is it called mirror and not fork ? A: Marketing picked the right word: it's a reflection — faithful, one-directional, and the mirror-world can never talk back.

Q: How is this different from the passive side of the house — like access_log tapping? A: access_log records what happened; mirror replays the request to a real service that can compute with it. One is observability (Lesson 12.1’s domain), the other is experimentation.

Q: Can the mirror target be HTTPS / another host? A: Any proxy_pass target works — including TLS upstreams (Module 6's client-cert machinery if the sink demands mTLS).

Takeaways

• mirror duplicates method + URI + body as a background subrequest; the response is discarded and failures are invisible to the client — that's the whole safety contract.

• Faithfulness comes from $request_uri in the mirror's proxy_pass; the mirror location is internal and fully configurable (headers, timeouts, caching, sampling).

• The canonical play is the dark launch: shadow v2, diff, then promote. The canonical sin is mirroring non-idempotent writes.

• Mirror traffic is real traffic: costs, PII, and memory double — sample with split_clients, strip secrets, mind the big bodies.