{"id":1451,"date":"2026-10-20T23:10:36","date_gmt":"2026-10-20T23:10:36","guid":{"rendered":"https:\/\/virtualserversvps.com\/blog\/?p=1451"},"modified":"2026-10-20T23:10:36","modified_gmt":"2026-10-20T23:10:36","slug":"nftables-set-performance-hash-sizing","status":"publish","type":"post","link":"https:\/\/virtualserversvps.com\/blog\/nftables-set-performance-hash-sizing\/","title":{"rendered":"nftables Set Performance: Hash Sizing, Timeouts and the Cost of Linear Rules"},"content":{"rendered":"
\n

nftables sets are the single biggest performance lever most firewall configurations never touch. A ruleset with 400 individual ip saddr<\/code> matches evaluates linearly; the same 400 addresses placed in a named set is a hash lookup with O(1) behaviour, and the difference on a busy server is measured in microseconds per packet at a point in the path where microseconds compound. This article covers how nftables set internals map to kernel memory, how to size the set hash table so lookups stay cheap under load, and how to verify the difference with pktgen<\/code>, iperf3<\/code> and per-rule counters. A firewall policy is one of the few tuning areas where a correctly provisioned VPS with dedicated CPU shares<\/a> also matters, because set lookups are pure CPU work with no I\/O to hide behind.<\/p>\n

Why Sets Are Faster Than Rules<\/h2>\n

Every netfilter rule is evaluated in order until a match. With n<\/code> rules the worst case is n<\/code> comparisons per packet, each touching a different memory location in the ruleset blob \u2014 cache-hostile, and the reason a 500-rule firewall shows visible CPU at 100k pps.<\/p>\n

A named set compiles to a hash table inside the kernel. Lookup is a hash plus a bucket walk, roughly constant regardless of element count. For 10,000 blocked prefixes the difference is not 10% \u2014 it is orders of magnitude in evaluation cost. The kernel likewise optimises intervals and concatenations, but the fundamental win is that all elements live in one contiguous, pre-sized structure rather than in a rule chain.<\/p>\n

<code># slow: one rule per address\ntable inet filter {\n  chain input {\n    type filter hook input priority 0; policy drop;\n    ip saddr 203.0.113.7   accept\n    ip saddr 203.0.113.19  accept\n    # ... 400 more\n  }\n}\n\n# fast: a set\ntable inet filter {\n  set allowlist {\n    type ipv4_addr\n    flags interval\n    elements = { 203.0.113.0\/24, 198.51.100.0\/24 }\n  }\n  chain input {\n    type filter hook input priority 0; policy drop;\n    ip saddr @allowlist accept\n    ct state established,related accept\n  }\n}<\/code><\/code><\/pre>\n
Set Types, Flags and What They Cost<\/h2>\n\n\n\n\n\n\n\n
Set type<\/th>\nLookup cost<\/th>\nMemory per element<\/th>\nUse for<\/th>\n<\/tr>\n
ipv4_addr<\/code><\/td>\nO(1) hash<\/td>\n~32 bytes<\/td>\nPlain address blocklists<\/td>\n<\/tr>\n
ipv4_addr<\/code> + interval<\/code><\/td>\nO(1) hash, O(log n) for ranges<\/td>\n~48 bytes<\/td>\nCIDR prefixes, ranges<\/td>\n<\/tr>\n
ipv4_addr . inet_service<\/code> (concat)<\/td>\nO(1) hash<\/td>\n~40 bytes<\/td>\nPer-IP port allowlists<\/td>\n<\/tr>\n
Anonymous set in a rule<\/td>\nO(1) hash<\/td>\nSame<\/td>\nShort fixed lists, not updatable at runtime<\/td>\n<\/tr>\n
vmap<\/code> \/ map<\/code><\/td>\nO(1) hash<\/td>\n~56 bytes<\/td>\nIP \u2192 rate limit or mark lookup<\/td>\n<\/tr>\n<\/table>\n
The interval<\/code> flag is the one people miss. Without it, a prefix like 10.0.0.0\/8<\/code> is stored as an exact address match and will never match 10.1.2.3<\/code>. With it, the kernel builds interval trees per bucket. There is a real cost: interval sets are slower to insert and delete, so an allowlist that changes thousands of times per second is better as a plain hash set of individual addresses.<\/p>\n
Sizing the Hash Table<\/h2>\n
The hash table size is set with size<\/code> at creation and grows automatically at 75% load, doubling each time. Auto-growth is correct but not free: each resize rehashes every element under a lock, which on a set with a million entries is a measurable stall. Pre-size sets that will hold a known large volume:<\/p>\n
<code>nft add set inet filter blocklist { type ipv4_addr\\; flags timeout,interval\\; }\nnft add set inet filter blocklist { type ipv4_addr\\; size 262144\\; }\n\n# check actual allocation and utilisation\nnft -j list set inet filter blocklist | jq '.nftables[1].set | {size, elements: (.elem|length)}'\ncat \/proc\/slabinfo | grep -E 'nft_set|kmalloc-2k|kmalloc-4k'<\/code><\/code><\/pre>\n
Sizing rule of thumb: allocate twice the expected steady-state element count. A threat-intelligence blocklist of 100,000 prefixes should be created with size 262144<\/code>; the extra memory is roughly 12 MB, which is nothing compared with the rehash stall it avoids.<\/p>\n
Timeouts and Dynamic Sets<\/h2>\n
Sets with flags timeout<\/code> support per-element expiry. This is the correct mechanism for auto-ban tools that add an address after a brute-force attempt and remove it later. Two operational details matter:<\/p>\n