10.2 TCP Proxying & Load Balancing

Advanced

What you will learn

1. proxy_pass in stream{} — the four-line TCP proxy.

2. Upstream pools with connection-granular scheduling: round robin, least_conn, weights, and the hash family.

3. Connect-time failure handling: retries, quarantine, and what the client actually experiences.

4. The knobs that matter at L4: proxy_timeout, connect_timeout, and TCP tuning passthroughs.

5. Reading a real balancing sequence, captured live.

1. The simplest possible proxy

server {
    listen       9200;
    proxy_pass   127.0.0.1:9060;
    proxy_timeout  30s;
}

That is a complete, production-shaped TCP proxy. On connect: NGINX opens a connection to 127.0.0.1:9060, then pumps bytes both ways until either side closes or the idle clock ( proxy_timeout) fires. No URI-part games here — proxy_pass in stream takes exactly one address (or an upstream group, or a variable — Lesson 10.5 uses that superpower). The Module 4 URI-semantics table simply doesn't apply: bytes are bytes.

2. Pools and connection-granular scheduling

Swap the address for a group and you're balancing — with Module 5's grammar, minus everything that needed to parse requests:

upstream tcp-pool {
    server 127.0.0.1:9060;      # b-tcp-echo-1
    server 127.0.0.1:9061;      # b-tcp-echo-2
}

upstream tcp-lc {
    least_conn;
    server 127.0.0.1:9062;      # b-tcp-echo-3
    server 127.0.0.1:9060;
}

server {
    listen     9201;
    proxy_pass tcp-pool;
}
server {
    listen     9202;
    proxy_pass tcp-lc;
}

The critical difference from http{} balancing: scheduling happens once per connection. A pool of 50/50 servers can sit 90/10 busy if connection lifetimes differ wildly — a database proxy where read connections close fast and report connections stay open for hours is the classic victim. Which is exactly why least_conn earns its keep here more than at L7: it balances live sessions, not arrival rates.

What survives intact from Module 5:

• round robin and weight= (relative connection share),

• least_conn, hash $variable [consistent], random two — for stream, hash keys come from binary facts like $remote_addr or $server_port, not application data;

• max_fails/ fail_timeout, backup, down — the whole failure vocabulary, with the same quarantine semantics (Section 3).

What doesn't survive: keepalive pools ( keepalive is meaningless — there's nothing to reuse when the session is the connection), and slow-start (arrival-based, not load-based).

The balancing sequence is deterministic — six fresh connections through :9201 alternate strictly (real capture):

conn 1 -> b-tcp-echo-1: lb
conn 2 -> b-tcp-echo-2: lb
conn 3 -> b-tcp-echo-1: lb
conn 4 -> b-tcp-echo-2: lb
conn 5 -> b-tcp-echo-1: lb
conn 6 -> b-tcp-echo-2: lb

And least_conn on :9202, where echo-3 + echo-1 sit in the pool — fresh connections land on the fewest-live-sessions peer, which for instantaneous echo sessions is simply "the other one" (real capture):

conn 1 -> b-tcp-echo-1: lc
conn 2 -> b-tcp-echo-3: lc
conn 3 -> b-tcp-echo-1: lc
conn 4 -> b-tcp-echo-3: lc

3. Failure handling, with receipts

Kill a backend and connect: three things happen, all visible in the logs. These receipts are from a real broken-backend window — the example TCP backends were down for a minute, and the logs caught the whole story.

1. Connect-time retry across the pool. The stream{} module retries the next peer when connecting fails (this is not a mid-session save — sessions don't migrate). The retry chain is right in the access log, comma-separated in $upstream_addr:

TCP 502 sent=0 received=0 session=0.000 upstream=127.0.0.1:9061, 127.0.0.1:9060

Two addresses, one failed session: 9061 was tried, refused, 9060 tried, refused — only then did the client get the bad news.

2. Quarantine, same as Module 5. After max_fails (default 1) connect failures inside fail_timeout (default 10s), the peer is parked:

[warn] *59 upstream server temporarily disabled while connecting to upstream,
       client: 127.0.0.1, server: 0.0.0.0:9201, upstream: "127.0.0.1:9061",
       bytes from/to client:0/0, bytes from/to upstream:0/0

3. Pool exhaustion is a 502 -shaped session. When every peer is parked, the client's connection is accepted and immediately closed — the session logs 502 with the group name as upstream:

[error] *62 no live upstreams while connecting to upstream,
       client: 127.0.0.1, server: 0.0.0.0:9201, upstream: "tcp-pool"
TCP 502 sent=0 received=0 session=0.000 upstream=tcp-pool

Note what the client experienced: TCP connect succeeded (NGINX answered — that's the listen socket), then instant close. There is no HTTP 502 page here; a database client sees "connection reset by peer." Lesson: at L4, a proxy outage looks like a backend outage unless you own the logs.

4. The knobs that matter at L4

The production pairing people get wrong: proxy_timeout shorter than the protocol's legitimate think-time kills healthy sessions (a nightly batch job with 15-minute compute gaps needs 1h, not the default), while longer than the firewall's idle drop means NGINX holds half-open sessions. The right value is a conversation with whatever sits between you and the client — the same negotiation you ran for keep-alive timeouts at L7 in Module 4.

5. Verify it yourself

Break things on purpose — the receipts in Section 3 came from exactly this:

kill $(cat /run/b-tcp-echo-2.pid)   # one pool member down
tail -n 20 /var/log/nginx/error.log               # quarantine receipts

Observe: with one peer down, four connections to :9201 all answer b-tcp-echo-1 — and the pool kept serving: $upstream_addr in the log names the single survivor, not a balanced pair.

Try it yourself

• Balancing probe: six fresh connections to :9201, one line each — read the alternation in stream.log

• Failure theater: kill a backend, watch error.log and stream.log, revive it

Q&A

Q: Why not just use a DNS record with several IPs? A: Because DNS gives you no failure detection, no quarantine, no weights — and clients cache. The pool is the value-add.

Q: least_conn at L4 balances live sessions — what balances bytes ? A: Nothing built-in; hash $remote_addr pins by client so heavy senders stay heavy on one peer, which is often what you want for cache locality anyway. True byte-based balancing is a metrics-loop problem (Module 12 gives you the metrics half).

Q: Can a session migrate backends mid-life? A: No. Not on failure, not on rebalance, not ever — the 4-tuple is the session. Migrating means terminating and reconnecting, and only the application protocol can negotiate that.

Takeaways

• Balancing moved to connect time: round robin/weights share arrivals, least_conn shares live sessions — prefer it when connection lifetimes vary.

• Connect failures retry across the pool ( proxy_next_upstream), log as comma-separated $upstream_addr chains, and quarantine peers exactly like Module 5.

• Pool exhaustion = accepted-then-closed connections logging 502 against the group name — clients see "reset by peer", so the proxy's logs are the only place the truth exists.

• proxy_timeout is the session idle clock — tune it against your protocol's think-time and your firewall's patience, in that order.