5.1 The upstream Context: Pools and Round Robin

Intermediate

What you will learn

1. upstream anatomy: the name, the server lines, and the parameters every peer accepts.

2. Plain round robin — how it actually behaves (worker state, not global counters).

3. Smooth weighted round robin — the 3:1 cycle, deterministic and fair.

4. Pool-level state: what reloads preserve and what they reset.

5. The zone directive (shared-memory pools) — why Module 13 needs it.

1. Anatomy of an upstream

upstream node-pool {                    # the NAME becomes a virtual host
    server 127.0.0.1:9001;              # a peer
    server 127.0.0.1:9002 weight=3;     # a peer with parameters
    server unix:/run/app.sock;          # unix sockets are peers too
    ...
}
# elsewhere:
proxy_pass http://node-pool/;           # referenced like a hostname

The name is not DNS — it's a config-level identity. NGINX resolves it to the peer list at load; every proxying/fastcgi/uwsgi/grpc directive can target it. Each server line accepts the parameter vocabulary you'll use all series:

The two example pools — the entire teaching surface of this lesson:

upstream pool-rr {                       # (1) plain round robin
    server 127.0.0.1:9001;
    server 127.0.0.1:9002;
    server 127.0.0.1:9003;
}
upstream pool-wrr {                      # (2) smooth weighted 3:1
    server 127.0.0.1:9001 weight=3;
    server 127.0.0.1:9002 weight=1;
}

2. Round robin — observed, not theorized

Six requests through /rr/, echoing which peer served each:

$ for i in $(seq 1 6); do
    curl -s -H 'Host: lb.example.test' http://127.0.0.1:8500/rr/x | jq -r .server
  done
b-node-2  b-node-3  b-node-1  b-node-2  b-node-3  b-node-1

Perfectly cyclic — for this sequence. Now the details that separate gurus from tutorial-followers:

• State is per worker. Each worker (Lesson 1.4) keeps its own rotation position. Six sequential curls on one connection hit one worker → clean cycle. Concurrent traffic spreads across workers → each continues its own cycle. The aggregate is balanced, not any single client's view.

• The cycle persists across requests but not across reloads of the upstream definition — a reload that keeps the block byte-identical keeps the state; editing peers resets it (and empties the stats).

• There is no queue. Round robin picks the next peer regardless of whether the previous one is busy — that's least_conn's job (next lesson). "Round robin" distributes choices, not load.

3. Smooth weighted round robin — the deterministic 3:1

Weighted round robin must solve a real problem: with weights 3 and 1, "three to node-1, then one to node-2" (block scheduling) would create bursts and uneven spacing. NGINX uses smooth weighted round robin (the same algorithm as classic LBs): interleave choices so the long-run ratio is exactly 3:1 but consecutive picks spread out.

Eight requests through /wrr/:

$ for i in $(seq 1 8); do curl -s … /wrr/x | jq -r .server; done | tr '\n' ' '
b-node-1 b-node-2 b-node-1 b-node-1 b-node-1 b-node-2 b-node-1 b-node-1

Count them: 6 × node-1, 2 × node-2 — exactly 3:1 over the 8 requests (a full cycle of 4 repeated twice: 1 1 2 1 | 1 1 2 1 — the tie at equal current_weight goes to the first-listed peer, so the light node lands third). Deterministic, fair, smooth. This determinism is why you can assert exact counts:

$ for i in $(seq 8); do curl -s …/api/ | grep -o 'node-[12]'; done | sort | uniq -c
   6 node-1                # weighted 3:1 -> 6 of 8 to the weight-3 peer
   2 node-2                #                2 of 8 to the weight-1 peer

4. Pool-level state and the zone directive

By default, peer state (rotation position, failure counters, connection counts) is per worker — Lesson 1.4's "no shared blackboard" law, applying to pools exactly as it did to everything else. Two workers maintain independent failure counters and independent keepalive caches.

upstream shared-pool {
    zone shared-pool 64k;          # shared memory: ONE copy of peer state
    server 127.0.0.1:9001;
    server 127.0.0.1:9002;
}

zone moves that state into a shared-memory zone: failure detection becomes fleet-wide (one worker's bad experience protects them all), and — the reason it exists — Plus's dynamic upstream API requires it to add/remove peers without a reload. Open source keeps zone useful for coherent failure counting; Module 13 revisits it as the foundation of dynamic reconfiguration strategies.

5. Reload semantics for pools

What a graceful reload (Lesson 1.5) preserves vs. resets:

That last row is the production pattern: down + reload = graceful drain. New requests stop choosing the peer; in-flight requests and keepalive connections wind down on their own clocks.

Try it yourself

1. Reproduce both distributions (§2, §3) — then break the cycle: fire requests concurrently ( seq 1 12 | xargs -P4 -I{} curl -s … /rr/x | jq -r .server | sort | uniq -c) and watch the aggregate stay balanced while no single thread sees a clean cycle.

2. Change the weights. Make it 5:1, reload, run 12 requests — predict the exact split before looking (smooth WRR: cycle of 6, 10:2 over 12).

3. Watch reload preserve state. Run 2 requests (mid-cycle), reload with an unchanged block, run 4 more — the cycle continues where it left off. Now add a comment to the block (forcing a "changed" peer set) and repeat: fresh rotation.

4. Drain a peer. Add down to one peer of pool-rr, reload, verify 6 requests touch only the survivors; remove down, reload, watch it rejoin mid-cycle.

Q&A

Q1. Why doesn't every client see the perfect 1-2-3-1-2-3 cycle?

Q2. Weights 5 and 1 — describe the smooth-WRR pattern over one full cycle.

Q3. What does adding zone change observably, in open source?

Q4. How do you drain one peer for maintenance with zero dropped requests?

Takeaways

• upstream turns names into pools; every *_pass directive can target one; peers take a rich parameter vocabulary ( weight, max_fails, fail_timeout, backup, down, resolve).

• Round robin is per-worker, stateful, and queueless — it distributes choices; smooth WRR distributes them at exact ratios with fair spacing (deterministic enough to assert on).

• zone shares peer state across workers and unlocks coherent failure detection (and Plus dynamics later).

• Reloads preserve sockets always and peer state when blocks are unchanged — and down+reload is the zero-drop drain pattern.