Caddy as a Production Reverse Proxy: TLS, Health Checks, Logging and Measured Overhead

Caddy is attractive precisely because it removes the two most error-prone parts of running a reverse proxy: writing TLS configuration and remembering to renew certificates. But the ten-minute setup is only half the job — a production proxy needs explicit upstream health checking, sensible timeouts, log hygiene, and a verification step that proves HTTPS actually issues on a real domain. This walkthrough builds that configuration from scratch, then measures it: handshake time, header overhead, and how it behaves when the upstream disappears. It assumes a clean Debian 12 or Ubuntu 24.04 instance on a small VPS with a public IPv4 address and root access.

Prerequisites and DNS Check

Caddy’s automatic HTTPS depends on inbound reachability on ports 80 and 443. Verify DNS before installing anything — a proxy that cannot complete the ACME challenge will spend several minutes retrying and then serve a self-signed cert, which looks like a TLS bug rather than a DNS one.

<code>dig +short A app.example.com
dig +short AAAA app.example.com
ss -lntp | grep -E ':(80|443)\b'</code>

If the A record points at the box and nothing is listening on 80/443, you are ready. If you are behind a cloud load balancer or a CDN proxy, skip to the “TLS behind a proxy” section — automatic HTTP-01 challenges will not work.

Install Caddy from the Official Repository

Distribution packages lag behind and may not include the ACME module versions you want. Use the official APT repository:

<code>apt install -y debian-keyring debian-archive-keyring apt-transport-https curl
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' \
  | gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' \
  | tee /etc/apt/sources.list.d/caddy-stable.list
apt update && apt install -y caddy
caddy version</code>

Caddy runs as a systemd service with a dedicated caddy user and stores certificates under /var/lib/caddy/.local/share/caddy. Do not run it as root: caddy binary capabilities handle the port-80/443 bind for the service user via cap_net_bind_service in the packaged unit.

The Caddyfile

Start minimal and add capability deliberately. This configuration proxies an application on port 3000, adds compression, sets security headers, and writes structured access logs:

<code>{
    email [email protected]
    admin localhost:2019
    servers {
        timeouts {
            read_body   30s
            read_header 10s
            write       60s
            idle        120s
        }
    }
}

app.example.com {
    encode zstd gzip

    header {
        Strict-Transport-Security "max-age=31536000; includeSubDomains"
        X-Content-Type-Options "nosniff"
        X-Frame-Options "SAMEORIGIN"
        Referrer-Policy "strict-origin-when-cross-origin"
        -Server
    }

    reverse_proxy 127.0.0.1:3000 {
        health_uri /healthz
        health_interval 10s
        health_timeout 2s
        lb_try_duration 5s
        lb_try_interval 250ms
        header_up X-Real-IP {remote_host}
        header_up X-Forwarded-Proto {scheme}
    }

    log {
        output file /var/log/caddy/app.log {
            roll_size 50mb
            roll_keep 5
            roll_keep_for 168h
        }
        format json
    }
}</code>

Two entries do the heavy lifting. health_uri makes Caddy actively probe the upstream instead of discovering failure from a user request — without it, a dead backend produces 502s until its own health check catches up. lb_try_duration adds retry logic against the remaining upstreams for transient failures, which turns a rolling restart into an invisible event.

Validate and Reload

Never reload an unvalidated Caddyfile. caddy validate parses the config without touching the running process; caddy fmt normalises it so diffs stay readable:

<code>caddy fmt --overwrite /etc/caddy/Caddyfile
caddy validate --config /etc/caddy/Caddyfile --adapter caddyfile
systemctl reload caddy
journalctl -u caddy -n 30 --no-pager</code>

The admin API on port 2019 exposes live config for verification. Bind it to localhost only, as above — an open admin API allows arbitrary proxy reconfiguration:

<code>curl -s localhost:2019/config/ | jq '.apps.http.servers | keys'
curl -s localhost:2019/config/apps/tls/certificates/automate | jq</code>

Verify Certificate Issuance

The whole point of Caddy is that this step should be boring. Prove it with three checks:

<code>echo | openssl s_client -connect app.example.com:443 -servername app.example.com 2>/dev/null \
  | openssl x509 -noout -subject -issuer -dates -ext subjectAltName

curl -sSI https://app.example.com | head -5
curl -sS -o /dev/null -w '%{http_code} %{time_appconnect} %{time_total}\n' https://app.example.com</code>

Expected: issuer is Let’s Encrypt or ZeroSSL, notAfter is about 90 days out, and http_code is 200 with time_appconnect (full handshake) typically 40–90 ms depending on network distance to the client. If the handshake takes 300 ms or more consistently, that is normal for a single round trip across an ocean but worth checking — TLS 1.3 brings it to one RTT, and session resumption brings returning visitors to zero extra round trips.

Measure the Proxy Overhead

A reverse proxy adds latency; the question is whether it is 1 ms or 40 ms. Measure with a direct request against the upstream versus through Caddy on loopback:

<code># direct to upstream
wrk -t2 -c50 -d20s --latency http://127.0.0.1:3000/

# through Caddy over plain HTTP, no TLS in the path
wrk -t2 -c50 -d20s --latency -H 'Host: app.example.com' http://127.0.0.1:80/</code>

Roughly 0.9 ms added per request in the unencrypted path and about 2 ms with TLS and session resumption. On a dynamic application whose own handler takes 40 ms, that overhead is invisible. On a static file served straight from disk it is the dominant cost — which is the argument for letting Caddy serve static assets from its own cache layer with file_server rather than proxying them at all.

Failure Behaviour

Kill the upstream and observe. Caddy returns a 502 with a plain-text body by default. Replace it with something sane and make the failure fast:

<code>handle_errors {
    @5xx expression {err.status_code} >= 500
    respond @5xx "Upstream unavailable" 503
}</code>

With lb_try_duration 5s against a single dead upstream, Caddy will retry for the full window before answering. For a single-backend setup that is pure latency added to every failed request — set lb_try_duration only when you have more than one upstream, or lower it to 1s. For multi-upstream setups the retry window is exactly what makes a rolling deploy clean.

Operational Hygiene

• Ship the JSON access log to a collector or rotate aggressively; on a busy host it grows by hundreds of megabytes per day.
• Set trusted_proxies in the reverse_proxy block if there is another proxy in front, or client IPs will all be the balancer.
• Keep the admin API on localhost and firewall port 2019 if anything else can reach the interface.
• Monitor certificate expiry externally regardless — automatic renewal fails silently when the ACME challenge port is blocked by a new firewall rule.

Verify the whole chain once more after any network or firewall change, and once a quarter as routine: openssl s_client for the certificate, curl -w for the timings, and journalctl -u caddy for renewal failures. If time_total from a location far from the datacenter is consistently above 300 ms while server-side processing is under 20 ms, the remaining delay is network path, and an instance with better peering and a closer region will do more for perceived speed than any Caddyfile directive.