4.2 The Header Contract: Making the Backend Whole

Intermediate

What you will learn

1. The canonical proxy header set — and what breaks without each line.

2. X-Forwarded-For chain semantics and $proxy_add_x_forwarded_for.

3. The other side of the glass: restoring the real client IP on NGINX( real_ip module) when IT is the proxied one.

4. proxy_http_version, hop-by-hop headers, and what NGINX strips.

5. The array-cancellation ambush, live: one custom header silently deleting the whole hygiene set.

1. The canonical header set

The production block (verbatim from the example gateway server):

proxy_set_header  Host              $host;
proxy_set_header  X-Real-IP         $remote_addr;
proxy_set_header  X-Forwarded-For   $proxy_add_x_forwarded_for;
proxy_set_header  X-Forwarded-Proto $scheme;

What each line restores for the backend:

The example's echo backend displays exactly what arrived. Three probes, three lessons:

$ curl -s -H 'Host: gw.example.test' http://127.0.0.1:8400/plain/ | jq .headers
{
  "host": "gw.example.test",
  "x-real-ip": "127.0.0.1",
  "x-forwarded-for": "127.0.0.1",
  "user-agent": "curl/8.14.1",
  …
}

Now with a forged upstream chain (you are the first hop):

$ curl -s -H 'Host: gw.example.test' -H 'X-Forwarded-For: 203.0.113.9' \
       http://127.0.0.1:8400/plain/ | jq .headers
{
  "host": "gw.example.test",
  "x-real-ip": "127.0.0.1",
  "x-forwarded-for": "203.0.113.9, 127.0.0.1",    ← appended, not replaced
  …
}

2. The other side of the glass: real_ip on NGINX itself

The header set above is for backends behind NGINX. But NGINX is often itself behind something — a cloud LB, Cloudflare, another proxy — and then $remote_addr is the middlebox, not the human. The real_ip module (compiled in: check your Lesson 1.3 nginx -V for --with-http_realip_module) restores truth at the edge:

# trust only these proxies to speak XFF:
set_real_ip_from  10.0.0.0/8;
set_real_ip_from  192.0.2.0/24;
real_ip_header    X-Forwarded-For;
real_ip_recursive on;      # walk the chain from the right: the last non-trusted address wins

# now, everywhere in this NGINX:
#   $remote_addr        = the real client
#   $realip_remote_addr = the proxy that actually connected (kept for logs)

real_ip_recursive semantics deserve ten seconds: with it off, the rightmost XFF entry is used (the nearest proxy's view); with it on, NGINX walks from the right, skipping every address it trusts via set_real_ip_from, and stops at the first untrusted address — the client (or the first lying hop; trust is a chain you vouch for).

Every later module that keys on $remote_addr — rate limiting (Module 7), geo (7), logs (12) — silently becomes wrong if you forget this on a fronted deployment. This lesson is why we teach it now.

3. proxy_http_version and hop-by-hop headers

By default NGINX speaks HTTP/1.0 to upstreams — a 1996 dialect chosen for safety. The modern setting:

proxy_http_version  1.1;

What 1.1 unlocks upstream: keep-alive connections (Module 5's keepalive pools — the single biggest latency win in proxy tuning), chunked transfer encoding (streaming!), and the WebSocket Upgrade (Lesson 4.4 requires this line). The example sets it in the WebSocket location only, so you can see the default's cost first: in access.log, upstream connections for /plain/ recycle per request.

Hop-by-hop headers — Connection, Keep-Alive, Upgrade, TE, Trailer, Transfer-Encoding, Proxy-* — belong to one connection, never to a request; NGINX strips them by spec when relaying. That's why Lesson 4.4 must re-add Upgrade semantics with proxy_set_header — the stripping is correct behavior, and the map-driven re-add is the sanctioned path through it.

4. The ambush: array-cancellation, live

Here is the example's /own-headers/ location — one innocent custom header, plus the same server-level hygiene set from §1:

# server level (the hygiene set of §1)…
location /own-headers/ {
    proxy_set_header  X-Custom  yes;      # ← the whole team dies here
    proxy_pass http://127.0.0.1:9001;
}

And what the backend actually received — the exact capture:

$ curl -s -H 'Host: gw.example.test' http://127.0.0.1:8400/own-headers/ | jq .headers
{
  "host": "127.0.0.1:9001",     ← hygiene Host CANCELLED; default took over
  "x-custom": "yes",            ← the only survivor: the location's own set
  "user-agent": "curl/8.14.1"
}

X-Real-IP? Gone. X-Forwarded-For? Gone. One proxy_set_header in the location cancelled every inherited one — Lesson 2.5's law, now wearing its most expensive costume. In production this surfaces as: "the new X-Canary header works, and mysteriously all our backend logging lost client IPs the same afternoon."

The professional fix is the Lesson 2.6 pattern — a snippet:

# snippets/proxy-headers.conf
proxy_set_header  Host              $host;
proxy_set_header  X-Real-IP         $remote_addr;
proxy_set_header  X-Forwarded-For   $proxy_add_x_forwarded_for;
proxy_set_header  X-Forwarded-Proto $scheme;

location /own-headers-fixed/ {
    include  snippets/proxy-headers.conf;    # re-declare, then extend:
    proxy_set_header  X-Custom  yes;
    proxy_pass http://127.0.0.1:9001;
}

Re-declaration, not inheritance — the only law-compliant way to add to an array set. (CI enforcement idea from Lesson 2.6 applies verbatim: nginx -T should show the snippet in every proxying context.)

5. The rest of the contract you'll actually need

One grab-bag of header directives you'll meet across the next modules — all follow the same inheritance law:

The hide/pass pair maps to a default denylist: NGINX relays all backend response headers except Date, Server, X-Pad, and X-Accel-... (the last family being Module 8/11's control channel — X-Accel-Redirect, X-Accel-Expires, X-Accel-Buffering — NGINX consumes them and hides them from clients by design).

Try it yourself

1. Reproduce §1's three probes. Verify the XFF append chain with two fake hops ( -H 'X-Forwarded-For: 198.51.100.7, 203.0.113.9').

2. Meet the ambush. Curl /own-headers/ and itemize what vanished. Then build the snippet fix (§4) under /etc/nginx/ as snippets/ + include; nginx -s reload; verify the fixed location shows the full set.

3. Watch HTTP versions. Change the echo probe: compare upstream behavior with proxy_http_version 1.1; added to /plain/ — then read Module 5's Lesson 5.4 when you want the connection-reuse proof.

4. Response-side surgery. proxy_hide_header X-Powered-By; on a location, then add a fake one to the echo's response via proxy_set_header… (it won't echo it back as a response header — use add_header on NGINX's side instead and observe both directions).

Q&A

Q1. Backend logs show client IPs as 127.0.0.1. Which two lines are missing, and where do they go?

Q2. NGINX behind a cloud LB: $remote_addr is the LB. Your rate limiter (Module 7) keys on it. Fix?

Q3. A teammate adds proxy_set_header Authorization "…"; to one location and the app's virtual-host routing breaks. Why?

Q4. Why is X-Forwarded-For a chain rather than a single address, and who is entitled to trust its leftmost entry?

Takeaways

• The four-line hygiene set ( Host, X-Real-IP, X-Forwarded-For, X-Forwarded-Proto) is the proxy's contract — put it in a snippet, include it everywhere you proxy, extend by re-declaring.

• X-Forwarded-For is append-built and edge-trusted; overwrite at your public boundary, append behind it.

• When NGINX is the proxied party, real_ip + set_real_ip_from + real_ip_recursive restore the true client into $remote_addr.

• proxy_http_version 1.1 unlocks upstream keep-alive, chunking, and WebSocket upgrades; hop-by-hop headers are stripped by spec and must be re-added deliberately.

• Response-side shaping lives in proxy_hide_header / proxy_pass_header; X-Accel-* is NGINX's private control channel (Modules 8 and 11 cash it in).