Nginx Upstream Keepalive: Cutting TCP Handshakes to Your App Server

By default, Nginx opens a brand-new TCP connection to every upstream (PHP-FPM, Node, Gunicorn, or a second Nginx) for every proxied request, then closes it. At 500 requests per second that is 500 handshakes and 500 teardowns per second — measurable as `TIME_WAIT` growth, rising p99 latency, and CPU spent in the kernel rather than in your application. Enabling upstream keepalive typically cuts end-to-end latency by 20–50% on connection-heavy workloads and drops ephemeral port pressure to near zero.

Measure the Problem Before You Fix It

Two independent signals. First, connection churn in the upstream’s own logs; second, `TIME_WAIT` sockets on the Nginx host.

# Load generator: 500 requests, 50 concurrent, against your Nginx
wrk -t4 -c50 -d30s --latency http://127.0.0.1/health

# In a second shell, count churn while the test runs:
watch -n1 "ss -tan | awk '{print \$1}' | sort | uniq -c | sort -rn | head -5"
# Before tuning you will watch TIME_WAIT climb into the thousands.
# Also check how many connections your upstream accepts:
ss -s | grep -i estab

If `TIME_WAIT` climbs monotonically during the test and only drains afterwards, you are handshaking per request. If it stays flat, keepalive is already in play somewhere and the latency you are chasing is elsewhere.

The Two-Directive Configuration

Upstream keepalive in Nginx needs a `keepalive` directive in the upstream block **and** a matching `proxy_set_header Connection` — the second one is what almost everyone forgets. The default `proxy_set_header Connection close;` in most example configs actively disables reuse.

upstream app_pool {
    server 127.0.0.1:9000;   # PHP-FPM, or 127.0.0.1:3000 for Node/Gunicorn
    server 127.0.0.1:9001;

    keepalive 32;            # idle connections kept per worker to this upstream
    keepalive_requests 1000; # reuse each connection this many times, then retire
    keepalive_timeout 60s;   # idle timeout for pooled connections
}

server {
    listen 443 ssl http2;
    server_name example.com;

    location / {
        proxy_pass http://app_pool;
        proxy_http_version 1.1;          # REQUIRED - 1.0 cannot do keepalive
        proxy_set_header Connection "";  # REQUIRED - clears the default 'close'
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        proxy_connect_timeout 2s;
        proxy_send_timeout    30s;
        proxy_read_timeout    60s;
        proxy_next_upstream error timeout http_502 http_503;
    }
}

Sizing the keepalive Pool

`keepalive N` is the number of **idle** connections retained per worker process, not a total cap. The correct N depends on concurrency, not on request rate:

Concurrent upstream requests per worker	keepalive value	Notes
< 10	16	small sites, cron-driven traffic
10–50	32	typical single-app VPS
50–150	64	busy API or WooCommerce checkout
> 150	128	also raise worker_connections

Each pooled connection costs one file descriptor and a kernel socket buffer — around 8 KB. Keeping 64 idle connections per worker on a 4-worker Nginx is 256 sockets, which is nothing. Do not set `keepalive 512` “just in case”; idle pooled connections to PHP-FPM still occupy a worker slot in the FPM pool, which is a real constraint on the other side.

Raise `worker_connections` in the `events` block to at least `keepalive_total + active_connections`, or Nginx will log `worker_connections are not enough` under load. For 4 workers with `keepalive 64` and a 1024 budget, `worker_connections 2048` is comfortable.

Verify That Reuse Is Actually Happening

Do not trust the config — count the sockets. After reloading, drive steady traffic and inspect the connection states on the upstream port:

sudo nginx -t && sudo systemctl reload nginx
wrk -t2 -c20 -d20s http://127.0.0.1/health &
sleep 3
# Count established sockets to the upstream (9000 here). Expect a stable, small number.
ss -tan state established '( dport = :9000 or sport = :9000 )' | wc -l
# And confirm TIME_WAIT is no longer growing:
ss -tan state time-wait | wc -l
wait

Before the change you would see the established count roughly equal to your concurrency and a `TIME_WAIT` count that keeps climbing. After it, the established count should settle at a number close to `keepalive × workers` (or your concurrency, whichever is lower) and stay there, with `TIME_WAIT` flat. That flat line is the whole optimization.

Benchmark the Difference

Run the same load test twice — once with the `Connection` header defaulting to `close`, once with the keepalive block enabled — and compare the latency distribution, not the average. Connection setup shows up in the tail:

# Before: comment out keepalive + proxy_http_version 1.1, then:
wrk -t4 -c50 -d30s --latency http://example.com/   | tee /tmp/before.txt
# Apply keepalive config, reload, then:
wrk -t4 -c50 -d30s --latency http://example.com/   | tee /tmp/after.txt
grep -E 'Latency|50%|90%|99%|Requests/sec' /tmp/before.txt /tmp/after.txt

On a loopback PHP-FPM setup with a real framework boot per request, the typical result is 15–35% lower p99 and a 10–20% throughput increase, with the gain shrinking to near zero once your application’s own work dominates the request. If p99 does not move, your bottleneck is inside the application, and the connection layer was never the problem.

The Failure Mode to Watch For

Long-lived pooled connections to a backend that restarts will produce sporadic 502s, because Nginx may hand a request to a socket the backend has already closed. `keepalive_requests 1000` bounds how long a connection lives, and `proxy_next_upstream error timeout http_502 http_503` retries the request on a fresh connection. Set both. If your upstream is PHP-FPM, check `listen.allowed_clients` and that the FPM pool is not configured with `pm = ondemand` and a low `pm.max_children` — pooled connections count toward that limit even when idle.

If you are benchmarking this against a new host, our VPS benchmark methodology explains how to keep load-test results comparable across machines, and the provider comparison tables show which plans have the CPU headroom to make upstream tuning visible in the first place. A single vCPU will saturate before connection reuse matters; two or more is where this pays off.

If you are tuning a small VPS and want hardware that does not fight you, our VPS provider performance tables break down CPU steal, NVMe IOPS, and RAM overcommit behaviour across the hosts we test on. Newer KVM nodes with dedicated vCPU pinning make the numbers in this article reproducible rather than aspirational. Two hosts we keep coming back to: InterServer VPS for flat-rate pricing with no RAM upcharge, and Cloudways managed cloud if you would rather not manage the kernel yourself. Compare the two against the benchmark methodology we publish before you commit.