9.1 Workers, Connections, and Limits

Intermediate

What you will learn

1. The three sizing knobs and their interaction: worker_processes × worker_connections vs worker_rlimit_nofile.

2. Reading the receipts: ps, the notice-level startup log, and /proc/<pid>/limits (one subtlety here is worth money).

3. The kernel ceilings above NGINX — and which ones you can touch without root (and which Module 16 shows you measuring).

4. worker_cpu_affinity, multi_accept, and when they matter.

5. A sizing procedure driven by measurements.

1. The three knobs

The main context (verbatim):

worker_processes       auto;      # one worker per CPU core
worker_rlimit_nofile   4096;      # each worker's file-descriptor ceiling

events {
    worker_connections  1024;     # per-worker connection slots
}

The interaction is arithmetic plus one subtlety each way:

connection capacity = workers × worker_connections
                       (× 1024 = 2048 sockets)

fd demand per connection ≈ 1 (client) [+1 upstream when proxied]
                          + listening sockets + log files + tmp files
                          + epoll internals

therefore: worker_rlimit_nofile  >  worker_connections × ~2  (proxied)

Bump worker_connections to 65 535 without raising rlimit_nofile and workers hit their fd ceiling mid-load — the classic [alert] worker_connections are not enough / too many open files pair that takes whole edges down during traffic spikes. The knobs are a set; tuning one alone is how incidents start.

2. Reading the receipts

The process table — workers exist and are sized right:

$ ps -o pid,ppid,args --pid $(cat /run/nginx.pid) --ppid $(cat …)
  PID  PPID COMMAND
 9468     1 nginx: master process …
 9469  9468 nginx: worker process     ← one per core
 9470  9468 nginx: worker process

The notice-level startup log — NGINX narrates its environment:

2026/09/22 21:45:53 [notice] 9467#9467: using the "epoll" event method
2026/09/22 21:45:53 [notice] 9467#9467: getrlimit(RLIMIT_NOFILE): 1024:100000

That getrlimit line is the master's view before applying your directives — a distinction that trips people up: it reads 1024:100000 (the shell inherited soft:hard limits) even though worker_rlimit_nofile 4096 was set, because the directive applies to workers, not the master. Which leads to the authoritative receipt:

/proc/<worker-pid>/limits — what the workers actually run under:

$ awk '/Max open files/{print $4}' /proc/$(pgrep -P $(cat …/nginx.pid) | head -1)/limits
4096                               # worker fd limit raised to 4096

Discipline for life: verify limits in /proc, not in config. The config is intent; /proc is truth. (Module 16 generalizes this into a full forensics toolkit.)

3. The ceilings above NGINX

Your worker_rlimit_nofile can never exceed the hard limit the master inherited (that 100000 in the log line — generous here, often 65535 or stingier in containers). Above that sit:

Run without root and the knobs left are the NGINX-side ones — which is honest: most NGINX-side tuning is possible without root; most kernel-side tuning is one config line for whoever owns the host. What you can always do is measure the ceilings (Module 16: ss -s, fd counts, SYN backlog drops) and translate them into tickets with numbers attached.

4. The exotic two: worker_cpu_affinity and multi_accept

worker_cpu_affinity  0001 0010;     # pin worker 0->cpu0, worker 1->cpu1

CPU pinning helps when co-tenants compete for cores (databases, JVMs): pin NGINX workers away from them (or vice versa). On a quiet, dedicated box, the scheduler already does better than most pin maps — measure (Module 16's flame graphs) before believing.

multi_accept  off;    # default: accept ONE connection per event-loop pass

on drains the accept queue in a burst per wakeup — better under connection storms, marginally worse for request fairness. NGINX's default ( off) is deliberately conservative; QUIC listeners (Lesson 9.5) require the socket-level sibling ( reuseport) regardless.

5. A sizing procedure (measure, then turn)

1. Measure current reality:
   - concurrent connections:  ss -tn state established '( sport = :443 )' | wc -l
   - per-worker fd usage:     ls /proc/<worker>/fd | wc -l   vs  limits
   - CPU: is the event loop idle or saturated? (pidstat / Module 16)
2. Capacity target = p99 concurrent × 2 (headroom) × ~2 fds each
3. Set:  worker_connections ≥ target / workers
        worker_rlimit_nofile ≥ worker_connections × 2 + slack
4. Verify in /proc after reload (the receipt discipline)
5. Only now file the kernel tickets (somaxconn, file-max) if numbers demand

The example numbers (2 workers, 1024 conns, 4096 fds) are deliberately modest — a box whose whole traffic is curl. The procedure is the deliverable: it scales to 64-core edges with 500k-connection ambitions unchanged.

Try it yourself

1. Collect all three receipts (§2): the process table, the notice-log getrlimit line, and the worker's /proc limits — and explain to yourself why the first two disagree about 4096.

2. Watch fd usage breathe. ls /proc/<worker>/fd | wc -l before, during, and after a keep-alive hammer ( curl in a loop, or 20 parallel from Module 5's tricks) — the count rises with live connections and settles back.

3. Break the arithmetic on purpose. Set worker_connections 8;, reload, and open 20 parallel connections to :8900 (use the slow backend through any location): some connections stall/refuse; the error log narrates. Restore.

4. Size for a scenario. "p99 = 5 000 concurrent, all proxied, 2-core box": compute the three knobs, apply, verify via /proc.

Q&A

Q1. worker_processes 8; worker_connections 16384; — the default rlimit_nofile (1024) unchanged. What happens under load?

Q2. The startup log says getrlimit: 1024:100000 but worker_rlimit_nofile 4096 is configured. Contradiction?

Q3. Why does upstream keepalive (Module 5) reduce pressure on a ceiling you can't tune without root?

Q4. When is worker_cpu_affinity worth its config line?

Takeaways

• The sizing set: worker_processes (cores) × worker_connections (slots) against worker_rlimit_nofile (ceiling ≈ 2× connections when proxying) — tuned together or not at all.

• Receipts over beliefs: ps for the family, notice-log for the environment, /proc/<worker>/limits for the truth.

• Kernel ceilings (file-max, somaxconn, port ranges) sit above it all —mostly root territory; your job is measuring and filing numbered tickets, plus NGINX-side workarounds (keepalive) where they exist.

• worker_cpu_affinity and multi_accept are situational refinements; QUIC's reuseport is not optional (Lesson 9.5).

• The sizing procedure is measurement-driven and identical from small boxes to 64-core edges.