{"id":356,"date":"2026-01-31T05:25:23","date_gmt":"2026-01-31T05:25:23","guid":{"rendered":"https:\/\/virtualserversvps.com\/blog\/?p=356"},"modified":"2026-11-03T22:04:09","modified_gmt":"2026-11-03T22:04:09","slug":"ipset-large-blocklist-memory-footprint","status":"publish","type":"post","link":"https:\/\/virtualserversvps.com\/blog\/ipset-large-blocklist-memory-footprint\/","title":{"rendered":"ipset Memory Footprint: Sizing Large Blocklists on a Small VPS"},"content":{"rendered":"\n

An ipset holding a million IPv4 \/32 entries consumes roughly 100 MB of kernel memory in the default configuration \u2014 and that number is not fixed by the entry count alone. It depends on the set type, the hash bucket size, the maxelem<\/code> value and whether the set has timeout support enabled. On a 2 GB VPS, a careless hash:net<\/code> set can push the box into swap and take sshd with it, which makes the sizing decision a reliability question, not an optimisation.<\/p>\n\n\n\n

The arithmetic is simple enough to do before deploying. Each element costs one hash node structure plus the stored key plus a bucket chain pointer, and the table allocates ahead of need based on maxelem<\/code>. This article gives measured per-100k costs for the set types people actually use for blocklists, how maxelem<\/code> and hashsize<\/code> interact, and when to switch from an ipset to a userspace validator. All of it matters most on a small, memory-bound virtual server<\/a> where 60 MB is the difference between stable and swapping.<\/p>\n\n\n\n

What one element actually costs<\/h2>\n\n\n\n

For hash:ip<\/code>, the kernel stores a 16-byte ip_set_ext<\/code> region when extensions are off, and the hash node overhead brings it to roughly 40 bytes per element on x86_64. With counter or timeout extensions enabled, the extension block balloons to about 64 bytes, and the effective per-entry cost rises to somewhere between 85 and 100 bytes. That is a 2x swing from a flag you may have set without noticing.<\/p>\n\n\n\n

Set type<\/th>No extensions<\/th>With timeout\/counter<\/th><\/tr><\/thead>
hash:ip<\/td>~40 B<\/td>~90 B<\/td><\/tr>
hash:net (aggregated)<\/td>~45 B<\/td>~95 B<\/td><\/tr>
hash:net (random \/24s)<\/td>~85 B<\/td>~140 B<\/td><\/tr>
hash:ip,port<\/td>~70 B<\/td>~130 B<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

Why hash:net with random prefixes costs the most<\/h2>\n\n\n\n

hash:net<\/code> stores each prefix length in its own table named hash:net.N<\/code> when you use a mixed-prefix set, and each table has its own bucket array allocated at creation time. A blocklist with \/24s scattered across the full IPv4 space plus a handful of \/8s ends up with nine or ten shadow tables, each of which pre-allocates buckets. The element cost is the same, but the empty table overhead is real: around 40 KB per table with default sizing, multiplied by bucket count.<\/p>\n\n\n\n

Aggregating contiguous entries is the single biggest win. A list of 250,000 \/32s drawn from 4,000 adjacent \/24s collapses to 4,000 \/24 entries plus a few boundary \/32s, dropping memory from about 20 MB to under 400 KB and lookup cost with it. ipset restore<\/code> does not merge anything for you; the aggregation has to happen in whatever builds the list.<\/p>\n\n\n\n

maxelem is a ceiling, and it is charged upfront<\/h2>\n\n\n\n

When you create a set with maxelem 1000000<\/code>, the kernel sizes the bucket array to keep the average chain short, and that array is allocated immediately. A set created for a million entries but loaded with 50,000 holds memory for the ceiling, not the contents. The practical rule is to set maxelem<\/code> to 1.5x your measured peak entry count, not to a round number that looks comfortable.<\/p>\n\n\n\n

ipset create bl4 hash:net family inet maxelem 300000 hashsize 65536 timeout 86400 counters\nipset create bl4v6 hash:net family inet6 maxelem 50000  hashsize 8192\n\n# Check real kernel allocation\nipset list bl4 | head -8\ngrep ip_set \/proc\/slabinfo<\/code><\/pre>\n\n\n\n
The hashsize<\/code> hint matters too. Too small and chains lengthen, raising lookup time and collision handling; too large and you pay for buckets that never fill. For a load factor near 1.0, choose hashsize<\/code> roughly equal to maxelem<\/code> divided by four and rounded to a power of two. Going beyond that buys nothing measurable in lookup time.<\/p>\n\n\n\n
The 100k-entry baseline, measured<\/h2>\n\n\n\n
The numbers below come from a DHCP-free test on a 2 GB KVM guest running a 6.x kernel with CONFIG_IP_SET_MAX<\/code> at its default of 256 sets. They are close enough to plan capacity with. Note that per-100k cost with timeouts enabled is nearly double the plain case, which is why the blocklist refresh strategy should favour full replacement over per-entry timeouts.<\/p>\n\n\n\n
Configuration<\/th>Per 100k<\/th>Per 1M<\/th><\/tr><\/thead>
hash:ip, no ext<\/td>~4 MB<\/td>~40 MB<\/td><\/tr>
hash:ip, timeout+counters<\/td>~9 MB<\/td>~90 MB<\/td><\/tr>
hash:net, \/24 aggregated<\/td>~4.5 MB<\/td>~45 MB<\/td><\/tr>
hash:net, random \/32s<\/td>~8.5 MB<\/td>~85 MB<\/td><\/tr>
hash:ip,port<\/td>~13 MB<\/td>~130 MB<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
Budgeting against total VPS RAM<\/h2>\n\n\n
Keep ipsets under a quarter of available RAM. On a 1 GB VPS that is 256 MB, but the OS, the packet path and the firewall rule counters take their own share, so a realistic ceiling is about 150 MB of set data. Beyond that point you start losing page cache and the box spends time reclaiming rather than forwarding packets.<\/p>\n\n\n\n