13.4 zone + resolve: Upstream Groups That Move

Advanced

What you will learn

1. The config: zone in the upstream, resolve on the server.

2. What the zone actually is (shared memory) and sizing.

3. Behavior receipts: same live flip, now with group semantics intact — plus what the log shows for a group.

4. The Plus API contrast: HTTP-driven membership vs DNS-driven.

1. The config

resolver  127.0.0.1:9053  valid=2s  ipv6=off;   # arms the runtime resolver

upstream dyn_pool {
    zone  dyn_pool  64k;                       # REQUIRED for resolve
    server  svc.example.test:9001  resolve;    # the magic parameter
}

server {
    listen      9501;
    location / {
        proxy_pass  http://dyn_pool;
    }
}

Two directives cooperate:

• resolve (on the server line) moves that name out of the config-load context (Lesson 13.2's gotcha — without it, nginx -t demands the system resolver know the name) and into the runtime resolver's care.

• zone allocates the shared-memory region where the group's runtime state lives: the resolved address list, peer stats, quarantine timers. Workers share one truth; a re-resolution in worker 3 is instantly visible to all.

Availability: server … resolve in OSS since 1.27.3 (released 26 November 2024); before that, this shape was Plus-only. On a current stable release (1.30.5 here) it's simply there — and it's the reason the "variable trick or nothing" era is over.

2. What the zone is, and sizing

The zone is a slab of shared memory sized by name, holding runtime state for the group. Sizing intuition:

• each peer entry + resolved addresses + stats ≈ a few hundred bytes;

• the example's 64k holds a fleet of dozens comfortably;

• undersizing fails loudly at start: zone "dyn_pool" is too small — a config-time error, not a runtime limp, so you find out in CI, not in prod.

Beyond membership, the zone has a second career: it's what makes max_fails/quarantine state coherent across workers (Module 5's "temporarily disabled" receipts were per-zone truth), and it's the precondition for Plus's runtime API mutating the group. One directive, three jobs: shared truth, resolved membership, API surface.

3. Receipts: the flip, with group semantics

Same experiment as Lesson 13.3 — record flipped .2 → .3 — now through the group door (real capture):

after the flip:
  :9501 -> b-node-dns-b        (followed, no reload)
  :9502 -> b-node-dns-a        (static contrast: never moved)

But look at the log line for a group request during the same window (real):

rsn="dyn_pool"                 <- upstream_addr for the FIRST request after a
                                  peer change: the GROUP name, because the
                                  resolved list was being replaced
rsn="127.0.0.2:9001"
rsn="127.0.0.3:9001"

That upstream_addr = "dyn_pool" line is real group anatomy: while the resolved list is mid-update, the request is assigned the group (and served by whatever the completed update yields) — evidence you're in group machinery now, not the variable pattern's single-address world.

What you KEEP from Module 5, now with runtime membership:

• weight= — relative share as addresses come and go;

• least_conn/ hash — scheduling across the current resolved list;

• max_fails/ fail_timeout — quarantine still works: a resolved address that refuses connections gets parked exactly like a static peer, and — key difference — when DNS stops returning it, it leaves the pool entirely rather than sitting parked.

And NXDOMAIN on the whole name still behaves like Lesson 13.3: requests fail honestly (502s) while the cache says "no such name," and recovery is automatic on restore. Both doors ( :9500 variable, :9501 group) recover within the same window.

4. The Plus API contrast

Plus's upstream API drives membership over HTTP instead of DNS:

GET  /upstreams/dyn_pool/servers       -> current members + state
POST /upstreams/dyn_pool/servers       {"server": "10.0.0.7:8080"}
DELETE /upstreams/dyn_pool/servers/10.0.0.5:8080
PATCH /upstreams/dyn_pool/servers/10.0.0.5:8080  {"drain": true}

The drain row is the one that genuinely hurts to lose in OSS: deploys that terminate instances want "stop sending new, let old finish," and DNS can't express it (a record is either there or not). OSS approximations: keep the old address serving during drain (the app returns 503-with-grace), or use a two-upstream cutover with backup — cruder, workable.

Try it yourself

• curl -s http://127.0.0.1:9501/ — hit it across a record flip and the group door follows; delete the record and it fails and recovers in lock step with the variable door (Lesson 13.3 has the receipts).

• Group semantics live: add least_conn; and a second resolved name to dyn_pool, flip one record, watch the access log's upstream_addr split across both fleets.

• Undersize the zone on purpose ( zone dyn_pool 1k;), reload — meet the config-time error that keeps this failure mode out of production.

Q&A

Q: Can a group mix resolved names and static IPs? A: Yes — server 10.0.0.9:8080; (static, pinned) alongside server svc.example.test:9001 resolve;. The resolved peer comes and goes; the static one never does. Mixed fleets (a managed service + your autoscaled app) look exactly like this.

Q: What happens to quarantine when an address disappears from DNS? A: It leaves the pool wholesale — cleaner than quarantine, which only parks it. If the address returns, it re-enters fresh (no lingering distrust), which is usually what you want and occasionally isn't (flapping fleets re-enter on probation every valid= — tune max_fails accordingly).

Q: zone name vs upstream name — must they match? A: No — zone upstreams_dyn_pool 64k; inside upstream dyn_pool is fine. Matching them is convention, not law. The zone name IS the shared-memory key: two upstreams sharing one zone name would collide (and nginx will tell you, loudly).

Q: Does resolve work in stream{} upstreams too? A: Yes — same parameter, same runtime resolver, Module 10's TCP/UDP pools can follow DNS just like these HTTP ones.

Takeaways

• The modern OSS shape: zone (shared runtime state, REQUIRED) + server … resolve (runtime-resolved membership) — group semantics and dynamic fleets together, since 1.27.3.

• The zone holds resolved lists + failure stats shared across workers; undersizing fails at config time, which is exactly where you want it to fail.

• Receipts: the group followed the same ~1.5 s flip; the log's upstream_addr="dyn_pool" mid-update line is group machinery showing itself; NXDOMAIN/restore behave like the variable pattern (honest 502s, automatic recovery).

• Plus's API adds instant HTTP membership and — the one that matters — draining; OSS reaches for DNS-shaped approximations.