nftables Set Performance: Hash Sizing, Timeouts and the Cost of Linear Rules

nftables sets are the single biggest performance lever most firewall configurations never touch. A ruleset with 400 individual ip saddr matches evaluates linearly; the same 400 addresses placed in a named set is a hash lookup with O(1) behaviour, and the difference on a busy server is measured in microseconds per packet at a point in the path where microseconds compound. This article covers how nftables set internals map to kernel memory, how to size the set hash table so lookups stay cheap under load, and how to verify the difference with pktgen, iperf3 and per-rule counters. A firewall policy is one of the few tuning areas where a correctly provisioned VPS with dedicated CPU shares also matters, because set lookups are pure CPU work with no I/O to hide behind.

Why Sets Are Faster Than Rules

Every netfilter rule is evaluated in order until a match. With n rules the worst case is n comparisons per packet, each touching a different memory location in the ruleset blob — cache-hostile, and the reason a 500-rule firewall shows visible CPU at 100k pps.

A named set compiles to a hash table inside the kernel. Lookup is a hash plus a bucket walk, roughly constant regardless of element count. For 10,000 blocked prefixes the difference is not 10% — it is orders of magnitude in evaluation cost. The kernel likewise optimises intervals and concatenations, but the fundamental win is that all elements live in one contiguous, pre-sized structure rather than in a rule chain.

<code># slow: one rule per address
table inet filter {
  chain input {
    type filter hook input priority 0; policy drop;
    ip saddr 203.0.113.7   accept
    ip saddr 203.0.113.19  accept
    # ... 400 more
  }
}

# fast: a set
table inet filter {
  set allowlist {
    type ipv4_addr
    flags interval
    elements = { 203.0.113.0/24, 198.51.100.0/24 }
  }
  chain input {
    type filter hook input priority 0; policy drop;
    ip saddr @allowlist accept
    ct state established,related accept
  }
}</code>

Set Types, Flags and What They Cost

The interval flag is the one people miss. Without it, a prefix like 10.0.0.0/8 is stored as an exact address match and will never match 10.1.2.3. With it, the kernel builds interval trees per bucket. There is a real cost: interval sets are slower to insert and delete, so an allowlist that changes thousands of times per second is better as a plain hash set of individual addresses.

Sizing the Hash Table

The hash table size is set with size at creation and grows automatically at 75% load, doubling each time. Auto-growth is correct but not free: each resize rehashes every element under a lock, which on a set with a million entries is a measurable stall. Pre-size sets that will hold a known large volume:

<code>nft add set inet filter blocklist { type ipv4_addr\; flags timeout,interval\; }
nft add set inet filter blocklist { type ipv4_addr\; size 262144\; }

# check actual allocation and utilisation
nft -j list set inet filter blocklist | jq '.nftables[1].set | {size, elements: (.elem|length)}'
cat /proc/slabinfo | grep -E 'nft_set|kmalloc-2k|kmalloc-4k'</code>

Sizing rule of thumb: allocate twice the expected steady-state element count. A threat-intelligence blocklist of 100,000 prefixes should be created with size 262144; the extra memory is roughly 12 MB, which is nothing compared with the rehash stall it avoids.

Timeouts and Dynamic Sets

Sets with flags timeout support per-element expiry. This is the correct mechanism for auto-ban tools that add an address after a brute-force attempt and remove it later. Two operational details matter:

• timeout is per element, but the garbage collector runs lazily — expired elements occupy memory until the set is walked. A set with millions of short-lived entries needs an explicit gc-interval.
• Default element timeout can be set at creation with timeout 1h; per-element overrides use { 203.0.113.7 timeout 10m }.

<code>nft add element inet filter blocklist { 203.0.113.7 timeout 15m }
nft -j list set inet filter blocklist | jq '[.nftables[1].set.elem[] | select(.expires != null)] | length'</code>

Benchmarking the Difference

Counters on the set rule give the packet rate actually evaluated; iperf3 provides the load. Measure the same traffic profile against a linear ruleset and a single set rule:

<code># add counters to both variants
nft add rule inet filter input ip saddr @allowlist counter accept
nft list chain inet filter input -a | grep counter

# load: many short flows force per-packet evaluation
iperf3 -c 203.0.113.200 -t 30 -P 32 -l 64

# CPU spent in the softirq path
mpstat -P ALL 1 5
perf top -C 2 -e cycles:k -- sleep 10 | grep -E 'nft|nf_'</code>

Results from a 4-vCPU instance with 400 addresses, 32 parallel 64-byte flows:

The throughput ceiling roughly doubles and the CPU cost drops by about three quarters. On a box that was already CPU-bound, that is the difference between an overloaded firewall and a comfortable one.

Expressions That Silently Defeat the Set

Three patterns push evaluation back into linear territory:

1. Negation without a set — ip saddr != 203.0.113.7 cannot be hashed. Write the positive set and drop by default policy instead.
2. Mixing set membership with unrelated conditions in one rule — ip saddr @allowlist tcp dport != 22 still evaluates the set, but the negation on the port side can force a second pass. Split into two rules and let the verdicts be explicit.
3. Very large anonymous sets — they compile fine but cannot be updated without replacing the whole ruleset, which is an atomic flush and reload of every chain. Named sets can be edited element by element with no disruption.

<code># atomic reload without a service restart
nft -f /etc/nftables.conf
# or a surgical add/delete
nft add element inet filter blocklist { 192.0.2.44 }
nft delete element inet filter blocklist { 192.0.2.44 }</code>

Putting It in Persistent Config

nft list ruleset output is valid input; save it and start from the service so a reboot restores the same sets:

<code>nft list ruleset > /etc/nftables.conf
systemctl enable --now nftables
cat /proc/net/netfilter/nf_log 2>/dev/null   # confirm hook registration</code>

Verify after a reload that set contents survived — a truncated save is the most common cause of a rule matching nothing after a reboot. Then re-run the benchmark once on the saved config, because the numbers above only hold if the ruleset in memory is the one you measured. Runway for this kind of CPU-bound network work is what separates a plan with modern cores and honest CPU allocation from one that throttles under packet load — the firewall is often the first place that difference becomes visible.