nftables Flowtables and Rule Counters: Measuring a Firewall That Costs Nothing at PPS Scale

A firewall that costs nothing per packet is not a marketing phrase, it is a measurable property of the datapath. A plain nftables ruleset with 40 rules in a chain evaluates all 40 for every packet. At 1.2 Mpps that is 48 million rule evaluations per second, and on a 2-vCPU guest it shows up as 30-40% CPU in %softirq before your application sends a byte. Flowtables move established flows out of the ruleset and hand them to the forward path with one lookup.

The trade-off is that you lose visibility: a packet that bypasses the ruleset also bypasses the counters you were using to prove the firewall worked. This covers how to measure the real cost of a ruleset at PPS scale, how flowtables change it, and how to keep accounting for both paths. Provider plans differ far more in packet rate than in core count — the options on our provider comparison list are worth checking before you size anything.

Counting Rules Costs PPS, Not Bandwidth

Ruleset cost is dominated by the number of rules a packet traverses, not packet size or link throughput. Small packets are the worst case: a 64-byte UDP flood delivers roughly 14 times as many packets per second as a 900-byte stream at the same bit rate, and every one of them pays the full rule walk. A ruleset invisible at 100 Mbit of TCP can saturate a vCPU at 200 kpps of DNS queries.

The threshold is easy to measure. Install a representative ruleset, generate traffic at a known packet rate, and compare against an empty chain. Anything above 8-10% of a core at your target rate deserves attention, because that is CPU unavailable to the application on a small plan.

# traffic rate and per-CPU split during a test
awk '/eth0/ {print "rx_pkts=" $3}' /proc/net/dev
mpstat -P ALL 2 3 | tail -6
nft list ruleset | grep -c '^		'     # rules actually walked

Rule Counters Are Per-CPU and Skewed

Counters on a rule are not a global integer. nftables allocates a per-CPU counter and sums them on read, so nft list ruleset under load reports a value that is stale by the time it prints. On a 2-vCPU guest with 60 kpps of matched traffic a counter can lag by thousands of packets, and two reads seconds apart can appear to move backwards.

More importantly, counters increment only on the slow path. Once a flow is in the flowtable its packets never reach the rule, so the counter freezes at whatever value it held at offload. A dashboard showing 4,000 matches per minute that flatlines is not evidence traffic stopped; it is evidence the rule stopped being consulted.

What Flowtables Actually Do

A flowtable is a hash table keyed by the 5-tuple, holding conntrack state plus the forwarding decision. The first packet takes the full path: conntrack, ruleset, verdict. If the verdict is accept and the flow qualifies, the kernel installs an entry and later packets in that direction match the flowtable before the ruleset is consulted at all.

Offload is unidirectional by design. Only the direction stated in the flowtable ruleset is accelerated; the reverse direction still traverses the ruleset, deliberately, so reply traffic stays filterable. A symmetric TCP workload gets roughly half its packets offloaded, and the CPU saving scales with that half, not with total traffic.

table inet filter {
    flowtable ft {
        hook ingress priority 0
        devices = { eth0 }
    }
    chain forward {
        type filter hook forward priority filter; policy drop;
        ip protocol { tcp, udp } flow add @ft
        ct state established,related accept
    }
}

When Flowtables Do Not Help

Flowtables accelerate forward traffic only. A VPS that terminates connections locally — web server, game server, VPN endpoint — never traverses the forward hook, so adding a flowtable changes nothing. This is the most common mistake: copying a router example onto a host that does not route, then wondering why PPS cost did not move.

Anything needing per-packet inspection defeats the purpose. Rate limiting with limit, QoS marking, and any rule carrying log must see every packet, and offloading a flow silently skips them. If your ruleset shapes rather than filters, flowtables are the wrong tool.

Scenario	Flowtable useful?	Expected CPU saving
Routed NAT gateway, TCP	Yes	50-70% of softirq time
Routed UDP relay	Yes, with short timeout	40-60%
Local web server	No	0%
Rate-shaping or marking	No	0%, rules skipped
Per-flow logging	No	0%, logs lost

Restoring Visibility Without Killing the Fast Path

You cannot count packets that bypass the chain, so counters must come from elsewhere. Fix the measurement before tuning anything: a single mpstat sample means nothing, so collect at 10-second intervals for an hour and compare p95, not the mean. The flowtable reports occupancy per CPU, which tells you how many flows are offloaded even without per-packet detail. For packet accounting, read interface counters — updated in the driver, untouched by any netfilter decision.

For security signal, add a low-priority chain that logs only new and untracked packets. Those are exactly the packets that still traverse the ruleset, and the ones you care about. Established flows accepted once do not need a log line on every match.

nft list flowtable inet filter ft -a      # per-flowtable size
nft list chain inet filter forward        # slow-path counters
conntrack -C; conntrack -S                # global conntrack stats
ip -s link show eth0                      # driver-side counters

Tuning Timeouts and Table Size

Each entry holds a timeout. TCP established defaults to 3600 seconds, far too long for a relay carrying short HTTP connections — the table fills with dead entries and hash lookup degrades. Drop it to 300 seconds for TCP and 30 for UDP unless flows genuinely need an hour.

Entry count matters more than anything else. A flowtable sized for 65,536 entries is fine for a few thousand concurrent connections and spills into random eviction at ten times that. Check occupancy at peak, and remember conntrack has its own limit that bites first.

Setting	Default	Relay recommendation
tcp established timeout	3600 s	300 s
udp timeout	30 s	30 s, 120 s for games
flowtable entries	65536	4x observed peak flows
nf_conntrack_max	varies	2x flowtable entries
rules in forward chain	—	under 20, sets over chains

Measure Before and After, Not After Only

The only defensible claim is a before-and-after at a fixed packet rate. Push a known 300 kpps of small TCP flows through the relay, record %softirq from mpstat -P ALL over 30 seconds without the flowtable, then repeat with it. Keep generator and rate identical; changing ruleset and load together gives numbers you cannot attribute.

If the second run shows no change, the likely causes are that traffic is not forwarding, the verdict is drop instead of accept, or the flowtable is attached to the wrong device — all ruleable out in a minute with nft list ruleset and conntrack -S. When it works, the savings are real and repeatable, which is also the point at which more cores stop being a substitute for a smaller ruleset. Check the VPS feature breakdown so you buy packet headroom rather than idle cores.