10.4 UDP Proxying

Advanced

What you will learn

1. What UDP proxying is for, and why "load balancing UDP" is a strange sentence.

2. The synthesized session: how NGINX turns a client's datagram stream into something with a lifetime.

3. The config — listen … udp, proxy_timeout, proxy_responses, and the return-path problem they solve together.

4. Real captures: datagram round-trips and per-"session" balancing.

5. The datagram traps: partial reads, and why your test tooling can lie to you.

1. Why proxy UDP at all

The flagship use case is DNS: a fleet of resolvers behind one anycast-ish address, with health checking and quarantine — L7 tools can't help because DNS-over-UDP has no connections to inspect. Beyond DNS: syslog/RELP-style collectors, QUIC fronting (Module 9's :8943 QUIC listener is UDP — an L4 UDP proxy can steer it), RADIUS, game telemetry.

The strange part: load balancing assumes units of work to assign, and units assume state. UDP has no connections, no session IDs, no "end". So the stream module does the only thing possible — it synthesizes state from the 4-tuple:

The first datagram from a client (source IP:port) to your listener creates a session pinned to one upstream. Subsequent datagrams from the same 4-tuple, within the session window, follow the same pin. After proxy_timeout of silence, the session evaporates and the next datagram re-rolls the dice.

This is exactly the NAT table trick your home router plays. The consequences reach everywhere:

• A DNS client sending 3 queries from one socket in 10ms hits one upstream, guaranteed — pinning is what keeps a fragmented protocol coherent.

• A client that pings every 11s with proxy_timeout 10s re-balances every single time and may see inconsistent backends — the "flapping" mystery that reads like a bug and isn't.

2. The synthesized session, in the log

The example UDP servers ( :9210 single backend, :9211 pool of two), and the exact config line that defines session life:

server {
    listen       9210 udp;
    proxy_pass   127.0.0.1:9070;
    proxy_timeout 10s;          # <- the ENTIRE session definition
}

Now the receipt that makes the model visible — two consecutive UDP sessions from the capture run (real stream.log lines):

127.0.0.1 [23/Sep/2026:00:38:10 +0000] UDP 200 sent=19 received=4 session=10.005 upstream=127.0.0.1:9070
127.0.0.1 [23/Sep/2026:00:38:11 +0000] UDP 200 sent=19 received=4 session=10.000 upstream=127.0.0.1:9070

Look at session=: 10.005 and 10.000 seconds for a single 1-datagram exchange. Nothing was transmitting for ten seconds — the session sat alive waiting for more datagrams until the timeout reaped it. Compare with the TCP echo sessions from Lesson 10.1 ( session=0.000) where connection close ends the story instantly. UDP sessions end by idle timeout or by the proxy_responses cap — no close() ever arrives from either side.

And the balancing story — six datagrams through the pool at :9211, sent with enough spacing to let each session expire (real capture; each line a fresh dice-roll):

dgram 1 -> b-udp-echo-1: d1
dgram 2 -> b-udp-echo-2: d2
dgram 3 -> b-udp-echo-1: d3
dgram 4 -> b-udp-echo-2: d4
dgram 5 -> b-udp-echo-1: d5
dgram 6 -> b-udp-echo-2: d6

Contrast: fire all six rapidly from one socket and the pin holds — all six answers come from one backend. Both behaviours are correct; which one you see depends on whether the client's socket outlives the timeout window.

3. The return-path problem, and proxy_responses

TCP proxies have it easy: the socket pair defines where answers go. For UDP, NGINX must keep a socket open toward the client to receive replies from the upstream — which is also how amplification attacks work. Hence the guard rails:

Production DNS fronting, assembled:

upstream resolvers {
    server 10.0.0.11:53;
    server 10.0.0.12:53;
}
server {
    listen      53 udp reuseport;
    proxy_pass  resolvers;
    proxy_timeout 5s;
    proxy_responses 1;
}

4. The datagram traps

Trap 1 — datagrams are atomic, and your tools may not be. A classic gotcha when scripting a UDP probe helper: writing it in pure bash ( /dev/udp + read -t 3) reads back exactly one byte — b — from a 19-byte reply, every time. Root cause: bash’s read -t pulls the socket one byte at a time, and a 1-byte recv() on a UDP socket consumes and truncates the entire datagram — the other 18 bytes are gone forever. A TCP stream doesn’t care about byte-at-a-time reads; UDP does, murderously. The fix is a 3-line Python helper using recvfrom(), which fetches datagrams atomically. If you ever script UDP health checks in bash, this is your gotcha — and it’s also why dig / ncat -u exist.

Trap 2 — "no listener" looks like "no answer." An unbound UDP port doesn't refuse (no RST exists in UDP); it silently drops, and ICMP port-unreachable may or may not surface as ECONNREFUSED on a connected socket only. Half the "UDP is unreliable" war stories are actually "my tooling couldn't see the error."

Trap 3 — MTU. The proxy forwards what it gets; fragmentation is the kernel's business. But proxy_responses-capped sessions plus fragmented 4k DNS answers over a 1500 MTU path is where "some queries fail" mysteries are born. Keep DNS answers small ( minimal-responses-style thinking) when a stream proxy sits in front.

5. Verify it yourself

Feel the session model by hand — one socket, rapid fire (all answers from ONE backend: the pin), versus spaced-out queries from fresh sockets (rotation). A one-line python3 loop over socket.SOCK_DGRAM can send each datagram from a fresh socket — which is how the §2 capture was made.

Try it yourself

• nginx — UDP needs no extra setup; the listeners show as UNCONN in ss -uln

• Probe: python3 one-liner with socket.SOCK_DGRAM (reusing the recvfrom() helper idea from Trap 1)

• Watch /var/log/nginx/stream.log for the session=10.00x timeout receipts

Q&A

Q: How do I health-check UDP backends? A: max_fails/ fail_timeout count ICMP-unreachable feedback (and Linux only surfaces it on connected sockets). There's no L7 health_check for arbitrary UDP protocols in OSS — a real DNS fronting deployment usually pairs the proxy with an external probe that flips down on peers via a config reload (Module 13's automation pattern).

Q: Can one server{} handle both 53/tcp and 53/udp ? A: Yes — two listen lines, one for TCP (DNS-over-TCP for big answers) and one udp. They share the proxy_pass, not the sessions.

Q: Why is my UDP "session" logging sent=0 ? A: Zero bytes forwarded to the client — typical when proxy_responses capped before the answer arrived, or the session timed out mid-exchange. Your reply came late; nobody was listening anymore. Welcome to UDP.

Takeaways

• NGINX synthesizes UDP sessions from the 4-tuple — pinned to one upstream while active, re-rolled after proxy_timeout of silence. The timeout is the session.

• session=10.00x on one-datagram exchanges is normal and is your receipt that sessions live out their window.

• proxy_responses is the anti-amplification cap; reuseport is the multi-worker medicine; both are default-off and both matter.

• Datagrams are atomic: byte-at-a-time readers (bash read -t) truncate them silently — a classic gotcha, caught and documented the hard way. Test UDP with datagram-aware tools.