Linux Socket Memory Tuning on a VPS: tcp_mem, tcp_wmem and the Overcommit Trap

Every byte the kernel buffers for TCP has to come from somewhere. On a VPS with a fixed RAM ceiling, socket buffers compete directly with your application, your page cache, and your database. Three sysctls govern this: net.ipv4.tcp_mem (the global memory a single protocol may consume), net.ipv4.tcp_wmem/tcp_rmem (per-socket send and receive buffer bounds), and the overcommit policy that decides whether the kernel even honours the allocations it promises. Raising buffers without understanding the interaction is one of the most common causes of a VPS that performs well under test and dies under real concurrency.

What each knob actually bounds

tcp_rmem and tcp_wmem take three values each: minimum, default, maximum. The kernel auto-tunes the buffer between min and max based on the connection’s bandwidth-delay product, starting at the default. tcp_mem takes three pages counts: below the first, no pressure; above the second, the kernel begins pressure-reduction; at the third, it refuses new allocations. Many tuning guides set only the per-socket maximums to huge values and never touch tcp_mem, so the global ceiling caps them anyway or memory pressure starts early.

Read current values before changing anything

sysctl net.ipv4.tcp_mem net.ipv4.tcp_rmem net.ipv4.tcp_wmem
sysctl net.ipv4.tcp_moderate_rcvbuf
cat /proc/net/sockstat        # current TCP memory use in pages

/proc/net/sockstat line TCP: inuse ... mem N is the ground truth for how many pages are currently charged to TCP. Compare N against the first value in tcp_mem to see how close to pressure you run in production. Interpreting idle values tells you nothing — sample under load.

Sizing by connection count, not by copy-paste

Worst-case socket memory is roughly max_connections × (tcp_rmem_max + tcp_wmem_max). On a 1 GB VPS that number is trivially larger than RAM, which is why the automatic tcp_mem ceilings exist — they stop the kernel from over-allocating. Do not multiply maximums by connection counts and set tcp_mem to the result; you will either OOM or make the kernel throttle every connection. Instead, size tcp_mem as a fraction of usable RAM (a common starting point is the kernel’s own auto-tuning, roughly 4%, 8%, 16% of RAM in pages) and confirm with sockstat that steady-state use stays below the first threshold.

# example: cap TCP at ~12% of a 1 GB host (pages of 4 KB)
# 1 GB = 262144 pages; 12% ~= 31457 pages
sysctl -w net.ipv4.tcp_mem='8192 16384 31457'

The overcommit trap

Socket memory is kernel memory, but the pages backing user buffers come from the same pool. If vm.overcommit_memory=1 is set (a common, badly-copied recommendation for Redis), the kernel hands out allocations it cannot guarantee. Under a burst of new connections that overcommit lets through, the host runs out of real RAM and the OOM killer chooses a victim — often your database, not the process that caused the spike. Check:

sysctl vm.overcommit_memory vm.overcommit_ratio
grep -E 'CommitLimit|Committed_AS' /proc/meminfo

If Committed_AS approaches CommitLimit, you are one allocation burst from the OOM killer regardless of how generous your socket buffers are. Keep overcommit_memory at 0 (heuristic) unless a specific application documents a need, and then watch Committed_AS continuously. This is one reason a fixed RAM ceiling makes buffer sizing a capacity question: our performance VPS plans lists plans by memory tier so you can pick an instance large enough that kernel socket memory never competes with the application.

Bufferbloat versus memory pressure

Large socket buffers are not free wins. A connection that buffers hundreds of kilobytes in the kernel adds latency under congestion — the sender keeps pushing because the buffer still has room, even though the bottleneck link is already full. This is bufferbloat, and on a shared VPS uplink it manifests as elevated RTT for everyone on the host, not just the heavy connection. The modern answer is not larger buffers but smarter queue management: keep per-socket buffers close to the bandwidth-delay product of your actual paths and let AQM (fq_codel, cake) absorb the excess. Only raise tcp_rmem maximums when you have measured that the window is the limiter on a specific long-fat path, and then only for that service via per-socket SO_RCVBUF rather than a global sysctl.

Monitoring the values that matter

Set up a small baseline so you notice drift before an incident. Track four signals: TCP memory use from /proc/net/sockstat, retransmit and drop counters from nstat, Committed_AS versus CommitLimit from /proc/meminfo, and the memory-PSI stall counter. None of them is expensive to collect and together they tell the whole story: rising socket memory plus rising drops means the ceiling is too low; rising Committed_AS with flat socket memory means something else is leaking. A five-line collector writing to logs every minute will catch a slow leak long before it turns into an OOM kill. Treat buffer tuning as an iterative loop — change one value, observe for a full load cycle, then decide — not a one-shot configuration.

Verify the change helped

Measure before and after with concurrent connection load, not a single transfer. Track drops and retransmits as well as throughput:

ss -s
nstat -az | grep -E 'TcpRetransSegs|TcpExtListenDrops|TcpExtTCPBacklogDrop'

If ListenDrops and BacklogDrop rise with connection count, the problem is the accept queue, not buffer memory — increasing tcp_rmem will not fix it. Match the tuning to the symptom. If you are repeatedly hitting socket-memory ceilings, the honest answer may be that the instance is undersized; our VPS comparison page compares plans with the RAM headroom needed to buffer high connection counts without entering tcp_mem pressure. Set conservative defaults, instrument sockstat and Committed_AS, and raise buffers only when the data shows the kernel is the bottleneck.