Nginx Reverse Proxy on a VPS: Upstream Keepalive, Buffering and Timeouts That Actually Matter

A reverse proxy is the single highest-leverage piece of software on a small VPS. It terminates TLS, absorbs slow clients, and keeps a pool of persistent connections open to your application so that PHP-FPM, Node, or Gunicorn never has to pay for a TCP handshake. This guide skips the “apt install nginx” filler and focuses on the directives that change behaviour under load: upstream keepalive, proxy buffering, and timeout arithmetic that prevents the classic 502 and 504 cascade.

Why the default proxy config falls over

Out of the box, Nginx opens a new TCP connection to the upstream for every proxied request and closes it immediately afterwards. On a 2 vCPU VPS serving 300 requests/second, that is 600 extra socket operations and 300 TIME_WAIT entries per second. The upstream’s accept queue becomes the bottleneck long before CPU does.

The fix is an upstream block with keepalive, and the matching proxy_set_header Connection "" that stops Nginx from passing Connection: close to the backend.

upstream app_backend {
    server 127.0.0.1:9000;
    keepalive 32;          # idle connections cached per worker
    keepalive_requests 1000;
    keepalive_timeout 60s;
}

server {
    listen 443 ssl http2;
    server_name example.com;

    location / {
        proxy_pass http://app_backend;
        proxy_http_version 1.1;
        proxy_set_header Connection "";   # required for keepalive
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Sizing the keepalive pool

The keepalive value is per worker process, not global. With worker_processes auto on a 2 vCPU box you get 2 workers, so keepalive 32 means up to 64 idle backend connections. Measure the real concurrency first:

# active connections to the upstream, sampled over 10s
ss -tn state established '( dport = :9000 )' | wc -l

# verify Nginx is reusing backend sockets (TIME_WAIT should be near zero)
watch -n1 "ss -tn state time-wait '( dport = :9000 )' | wc -l"

A healthy keepalive pool shows a flat, low TIME_WAIT count. If TIME_WAIT climbs in step with request rate, the Connection "" header is missing or proxy_http_version is still 1.0.

Buffering: protect the backend from slow clients

Without buffering, a backend worker stays occupied for the entire duration of a slow client download. For dynamic sites the right posture is to buffer responses on disk up to a sane limit, then stream.

proxy_buffering on;
proxy_buffers 16 16k;        # 256 KB total per connection
proxy_buffer_size 16k;       # headers, must exceed largest response header
proxy_busy_buffers_size 32k;
proxy_max_temp_file_size 1024m;

# stream large downloads instead of buffering them
location ~* \.(mp4|zip|iso)$ {
    proxy_buffering off;
    proxy_request_buffering off;
}

Directive	Too low	Too high
proxy_buffer_size	502 “upstream sent too big header”	wasted RAM per idle conn
proxy_busy_buffers_size	client sees stuttering	memory growth on many conns
keepalive	TIME_WAIT churn	idle sockets held on backend

Timeout arithmetic that prevents 504s

Timeouts must be ordered from the outside in: the proxy must wait longer than the backend, so the backend’s error surfaces as a clean 502/500 rather than a truncated connection. A common production set:

proxy_connect_timeout 5s;
proxy_send_timeout    60s;
proxy_read_timeout    60s;
send_timeout          60s;

# make Nginx try the next upstream on a timeout, not just a 502
proxy_next_upstream error timeout http_502 http_504;
proxy_next_upstream_tries 2;

If your application legitimately needs 90 seconds for a report export, raise proxy_read_timeout only for that location rather than globally. Global long timeouts are how a single stuck request consumes a worker slot that thousands of fast requests needed.

Verifying the change

Reload without dropping connections and confirm the behaviour with a quick benchmark against the loopback:

nginx -t && systemctl reload nginx

# 5000 requests, 50 concurrent, against the proxied endpoint
wrk -t4 -c50 -d20s http://127.0.0.1/

# backend handshake churn should not scale with request count
nstat -az TcpExtTCPReqQFullDrop TcpPassiveOpens

If TcpPassiveOpens grows by roughly one per request, keepalive is not working. After the fix it should grow only as workers cycle.

TLS and CPU on a small VPS

Terminating TLS at the proxy is the right call, but RSA handshakes are expensive on a single vCPU. Prefer an ECDSA certificate, enable session resumption, and let the main site’s comparison pages guide your instance sizing — a proxy plus app plus database rarely fits comfortably under 2 GB.

ssl_protocols TLSv1.2 TLSv1.3;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 1d;
ssl_session_tickets on;   # or off if you rotate keys aggressively

# check resumption is actually happening
openssl s_client -connect example.com:443 -reconnect -no_ticket 2>/dev/null | grep -c "New,"

HTTP/2, WebSockets and upgrade headers

A reverse proxy that forwards plain HTTP/1.1 will silently break two things people rely on: HTTP/2 multiplexing to the client, and WebSocket upgrades to the backend. The first requires http2 on the listening socket (or http2 on; in newer Nginx), the second requires explicit upgrade headers and a long read timeout on the specific location.

# enable HTTP/2 on the TLS listener
listen 443 ssl;
http2 on;

# WebSocket upgrade location
location /ws/ {
    proxy_pass http://app_backend;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_read_timeout 3600s;   # long-lived sockets
    proxy_send_timeout 3600s;
    proxy_buffering off;        # never buffer a socket stream
}

Note the conflict: Connection "upgrade" defeats keepalive for that location. Keep application traffic and WebSocket traffic on separate location blocks so the general path can retain its persistent connection pool.

Real client IP behind a CDN or load balancer

When a CDN or an upstream load balancer sits in front of your VPS proxy, $remote_addr becomes the balancer’s address and every log line is useless for rate limiting or forensics. The real_ip module restores the true client address, but only from sources you trust.

# trust only the CDN edge ranges
set_real_ip_from 203.0.113.0/24;
real_ip_header X-Forwarded-For;
real_ip_recursive on;

# confirm the module is compiled in
nginx -V 2>&1 | tr ' ' '\n' | grep -i realip

Getting this wrong is a security issue, not just a logging one: if you blindly trust X-Forwarded-For from any source, a client can spoof its own IP and evade your limit_req rules. Restrict set_real_ip_from to the actual proxy ranges.

Observing the proxy under real traffic

# enable a status endpoint (restrict to localhost)
location = /nginx_status {
    stub_status;
    allow 127.0.0.1;
    deny all;
}

# read active/idle counts and accepted vs handled requests
curl -s http://127.0.0.1/nginx_status

If active connections sit near your worker_connections limit, or handled lags accepted, you are dropping connections and should revisit worker sizing rather than proxy timeouts.

Summary

• Upstream keepalive plus Connection "" eliminates per-request handshakes; verify with TcpPassiveOpens.
• proxy_buffer_size must exceed your largest response header or you get 502s that look like app bugs.
• Order timeouts so the backend fails first; use proxy_next_upstream to retry cleanly.
• Keep global timeouts short and override per-location only where genuinely needed.
• On low-RAM instances, see the managed vs unmanaged comparison before deciding to self-manage the TLS stack.