7.4 Traffic Splitting: split_clients and the Two-Line Canary

Intermediate

What you will learn

1. split_clients — hash-based, deterministic, percentage-precise splitting.

2. Why determinism beats randomness for experiments (and for rollouts).

3. Choosing the split key: user identity, cookie, request — the stability contract.

4. The canary pattern end-to-end: split → two upstreams → gradual promotion.

5. Sticky tiers vs re-hash churn, and the split_clients + map composition for multi-way splits.

1. The directive

split_clients "${remote_addr}AAA" $edge_bucket {
    25%     canary;        # 25% of the KEYSPACE lands here
    *       stable;        # everyone else
}

Mechanics: hash the key string (MurmurHash2), map the hash space onto your percentages — deterministically. Same key → same bucket, every time, forever (until you change the percentages or key). The trailing "AAA" in the example's key exists to demonstrate the craft: the string can be anything — ${remote_addr}AAA differs from plain $remote_addr, and changing it re-rolls everyone (a useful escape hatch and a surprising footgun, both covered in §4).

Live from a real run — deterministic per client, both buckets reachable:

$ curl -s … http://127.0.0.1:8700/bucket
edge_bucket=stable
$ curl -s … http://127.0.0.1:8700/bucket        # same client, same answer
edge_bucket=stable

$ for s in a b c d e f g h i j; do curl -s … "/flip?seed=$s"; done | sort | uniq -c
   5 heads
   5 tails                # the 50% split reaches both buckets over 10 seeds

The second split (keyed on $arg_seed) is your laboratory: a controlled 50/50 coin you can re-roll by varying the seed — the Try it yourself exercise below uses exactly this to prove both the reach and the determinism.

2. Why deterministic beats random

"Random 10%" and "deterministic 10%" differ in the ways that matter:

The flip-every-request property of naive randomness destroys experiments (a "user" who saw both variants is in both cohorts) and destroys caches. Deterministic hashing keeps cohorts disjoint and auditable — "which bucket was Alice in?" has an answer.

3. Choosing the key — the stability contract

The split is only as good as its key's stability:

The rules that fall out: user-facing experiments want user-stable keys (cookie/ID — the same identity contract as Module 5's session affinity); never split on anything unique per request when you mean cohorts; and know your ramp plan before choosing (§4).

4. The canary pattern, end to end

Splitting to a variable is only half the pattern — the variable routes:

upstream stable-pool {
    server 10.0.0.11:8080;
    server 10.0.0.12:8080;
}
upstream canary-pool {
    server 10.0.1.11:8080;      # the new version, small fleet
}

split_clients "${cookie_uid}AAA" $checkout_pool {
    10%     canary-pool;        # hash-space percentage == fleet share
    *       stable-pool;
}

server {
    location /checkout/ {
        proxy_pass http://$checkout_pool;   # Lesson 4.1's rule 4:
    }                                       # variables -> no URI transform
}

Promotion is a dial: 10% → 25% → 50% → * (delete the rule) — one line per step, nginx -t, reload. Because assignment is hash-based, each ramp keeps most users on their current side (the same consistent-hashing property as Lesson 5.2: growing the canary share takes from the stable side proportionally, without re-rolling everyone).

The operational checklist around it:

• Cache keys must include the bucket (Module 8): $edge_bucket in proxy_cache_key, or canary users eat stable users' cached pages (and vice versa).

• Observability must label the bucket (Module 12): access log $edge_bucket, backend metrics tagged with the variant — the comparison is the experiment.

• Rollback is a percentage: back to * and reload; no deploy, no restart.

• Edge case — the re-hash cliff: changing the key string (that "AAA" suffix) or restructuring percentage lines re-rolls everyone at once. Reserve key changes for "new experiment" moments, never mid-ramp.

5. Multi-way splits — split_clients + map

split_clients handles two ways naturally; for richer segmentations, compose (the Lesson 7.1 pattern again):

split_clients "${cookie_uid}AAA" $rollout_shard {
    10%    in-rollout;
    *      not-in-rollout;
}
map "$rollout_shard:$country" $variant {
    default                  v1;
    "in-rollout:US"          v2-us;       # rollout only in the US…
    "in-rollout:CN"          v2-cn;       # …with a localized variant…
    "not-in-rollout:XX"      hold-page;   # unsupported regions: parked
}

Percentage of the keyspace × segments of the population — feature flags as configuration, no if in sight.

Try it yourself

1. Find your bucket — and its stability: hit /bucket five times; then change your identity (a cookie won't exist in curl — use the /flip?seed= endpoint with your own seeds) and watch assignment follow the key.

2. The 50% census. Run /flip across 40 seeds and count heads/tails — should be near 20/20. Run across 400 and watch the law of large numbers tighten (the percentages are exact in expectation; per-sample they're binomial).

3. Ramp simulation. Change the split from 50% heads to 25% heads, reload, re-run your 40 seeds — most tails stay tails, heads shrink. The no-re-roll property, measured.

4. Build the canary. Implement §4 against the example backends (9001 = stable, 9002 = canary — the echo's server field is your variant label). Ramp it 10 → 50 → 100% with reloads, confirming the population moves monotonically.

Q&A

Q1. Why is split_clients on $request_id wrong for an A/B test but fine for "send 10% of traffic to the shadow backend"?

Q2. Mid-ramp from 10% to 25%, do existing canary users stay on the canary?

Q3. Your experiment's cache hit rate collapsed after enabling a split. Diagnosis?

Q4. How do you debug "which variant did user 8412 see Tuesday?"

Takeaways

• split_clients: hash the key, partition the space by percentages —deterministic, auditable, rampable; both reachability and stability proven live above.

• Deterministic cohorts beat per-request randomness for every user-facing purpose; per-request assignment belongs to load shaping only.

• The key carries the stability contract: user IDs/cookies for cohorts, $request_id for volume, never confuse them.

• The full canary = split variable + variable-targeted proxy_pass+ bucket-aware cache keys + bucket-labeled telemetry + rollback as a percentage.

• Multi-way segmentation composes with map — feature flags as configuration.