nftables conntrack Table Exhaustion: Diagnosing Dropped Connections Under Load

When a busy VPS starts dropping connections that have nothing obviously wrong with them — no CPU spike, no memory pressure, plenty of bandwidth — the culprit is usually the conntrack table. Connection tracking is the kernel subsystem behind nftables’ stateful matching, and it has a fixed maximum size. Once that table fills, nft rules with ct state established,related stop matching, and packets that used to be accepted are dropped silently by the default policy.

The failure mode is nasty because it looks like a network outage, not a resource limit. Existing connections may survive for a while, new ones fail, and monitoring that only watches CPU and bandwidth shows nothing wrong. This article covers how the table fills, what the log evidence looks like, and the sysctls that actually fix it.

What the conntrack Table Actually Holds

Every connection that passes through a stateful rule gets an entry in the nf_conntrack table. Each entry tracks the tuple in both directions plus protocol state, timeouts, and helper information. On a 64-bit kernel, a typical TCP entry costs roughly 300 bytes of slab memory, so a 65,536-entry table consumes about 20 MB — trivial. The cost is not memory; it is the hard cap on simultaneous tracked connections.

Critically, nf_conntrack_max counts entries, and entries live until they time out, not until the socket closes. A web server handling 500 requests per second with a 5-second average response time needs roughly 2,500 live entries. Add a TIME_WAIT timeout of 120 seconds and the same server can hold 60,000 entries just from finished connections. This is why the table fills during bursts even when concurrency looks reasonable.

Reading the Counters Before the Table Fills

nft list ruleset will show your rules but not capability. Start with sysctl net.netfilter.nf_conntrack_count and net.netfilter.nf_conntrack_max. A count above 70% of max during normal peak is your warning that the next traffic spike or DDoS attempt will take down new connections. The ratio matters more than the absolute number, because it tells you how much headroom you have.

Also check nf_conntrack_expect_max if you use FTP, SIP or other helper-based protocols — expectations are a separate pool and can exhaust before the main table. On containers, remember that Docker and podman insert their own rules and can create large numbers of entries from SNAT if you have many short-lived container-to-container connections.

Log Evidence: What Exhaustion Looks Like

The kernel emits a rate-limited message when it drops a packet because the table is full. In dmesg or journald you will see lines like nf_conntrack: nf_conntrack: table full, dropping packet, sometimes with a source and destination. If the rate limiter has already fired you may see only nf_conntrack: nf_conntrack: table full, dropping packet (rate limit exceeded), which is why you should also raise net.netfilter.nf_conntrack_log_invalid only temporarily when debugging.

Complement this with nftables counters. Put a named counter on your input chain’s default drop rule: counter drop. If the drop counter climbs in lockstep with conntrack exhaustion messages, you have confirmed the mechanism. If the drop counter climbs but conntrack messages are absent, you have a different problem — usually a rule ordering bug or a missing ct state established accept.

Sysctl Reference

ParameterDefaultSuggested on 2–4 GB VPSEffect
net.netfilter.nf_conntrack_max65536 or RAM/16384·1024131072Table size cap
nf_conntrack_tcp_timeout_established432000 (5 days)3600Frees idle established entries
nf_conntrack_tcp_timeout_time_wait12030Biggest single win for web servers
nf_conntrack_tcp_timeout_close_wait6015Frees half-closed connections
nf_conntrack_tcp_timeout_fin_wait12030Same
nf_conntrack_udp_timeout3030Do not lower below DNS retry interval
nf_conntrack_udp_timeout_stream12060Long-lived UDP flows
nf_conntrack_bucketsmax/432768Hash table size; set at boot only
nf_conntrack_log_invalid00 (1 when debugging)Logs invalid packets

Note that nf_conntrack_buckets is not writable at runtime — it is set at module load, so it belongs in /etc/modprobe.d/nf_conntrack.conf as options nf_conntrack hashsize=32768. Setting the max without resizing buckets gives you a longer hash chain per bucket, which degrades lookup performance exactly when you are already under stress.

Lowering tcp_timeout_time_wait from 120 to 30 seconds is the highest-leverage change for a typical web VPS. It cuts steady-state table occupancy by a large fraction on short-lived HTTP connections, with the only risk being a slightly higher chance of receiving a stale duplicate segment that gets rejected — acceptable in practice for most workloads.

Reducing Entries Instead of Raising the Limit

Raising nf_conntrack_max is easy and often correct, but it treats the symptom. If you have a reverse proxy in front of your application, every proxied connection creates two entries — client-to-proxy and proxy-to-backend — so table usage is roughly double what direct traffic would produce. Keeping proxy-to-backend connections alive with keepalive pools eliminates the churn entirely and is usually worth more than any sysctl.

Second, make sure your ruleset does not force tracking where it is unnecessary. A rule like tcp dport 80 accept still creates a conntrack entry because nftables tracks by default when the chain is part of a stateful setup. If you have a pure stateless allow-list for a public port with no NAT, the notrack statement avoids the entry. Use it selectively — it disables stateful matching for that flow, so only apply it where you genuinely do not need it.

Applying the Fix Without Dropping Traffic

Write the sysctls to a file under /etc/sysctl.d/ so they survive reboot, then apply with sysctl --system. nf_conntrack_max and the timeout values are writable at runtime and take effect immediately; you can raise the max while connections are live without disturbing existing entries. Do not flush the table on a production box — conntrack -F will break every established connection, including your SSH session.

Verify with sysctl net.netfilter.nf_conntrack_max and watch nf_conntrack_count for a full traffic cycle before deciding you are done. A table that sits at 30% of the new maximum at peak is healthy; one that still hits 80% means your traffic pattern needs architectural attention, not another doubling. If you are choosing an instance for a connection-heavy workload, the VPS configurations sized for high connection counts matter more than raw CPU. For background on how instance sizing interacts with kernel limits, see the cloud VPS benefits guide.

Monitoring So It Never Surprises You

Export nf_conntrack_count and nf_conntrack_max as a ratio, and alert at 70% sustained for five minutes. Alert separately on the appearance of table full in the journal, because that message means drops have already started and the ratio alert may have been missed during a fast spike. Two alerts, one ratio and one event, covers both gradual growth and sudden bursts.

The same telemetry tells you when to act on nf_conntrack_expect_max or when a container runtime has started leaking entries. Connection tracking is one of the few kernel subsystems where a single ratio fully describes health, which makes it cheap to monitor and easy to justify to whoever owns the pager.

Leave a Reply