Tuning the Linux OOM Killer and systemd-oomd on Memory-Pressured VPS Instances

The OOM killer is the least understood thing on a small VPS. It fires, kills your database instead of the runaway process, and the only trace is a line in the journal you scroll past. Getting it right is not about “disabling” it — it is about controlling which process dies, and doing it before the kernel has to make a knife-edge decision. This guide tunes both the classic kernel OOM killer and the newer systemd-oomd, which work together and can also fight each other if misconfigured.

These techniques matter most on a small VPS where RAM is the binding constraint, and they complement the memory-reclaim tuning documented on the main site.

Understand the two layers

There are two independent mechanisms:

• Kernel OOM killer — fires when the kernel is genuinely out of reclaimable memory. It picks a victim by oom_score. This is a last resort that you cannot fully prevent, only steer.
• systemd-oomd — watches PSI (pressure stall information) and kills a whole cgroup before the kernel OOM killer activates. It targets the cgroup that is causing pressure, which is usually a much better choice than the kernel’s heuristic.

Check whether oomd is running before tuning anything:

systemctl status systemd-oomd
cat /proc/pressure/memory

/proc/pressure/memory is the same signal oomd acts on: some avg10 is the percentage of time at least one task stalled on memory; full avg10 is time everything stalled. Sustained full above a few percent means you are in the danger zone.

Steering the kernel OOM killer

Every process has an oom_score_adj from -1000 (never kill) to +1000 (kill first). The kernel computes an oom_score from memory usage and then biases it by this value. Protect your critical database:

# For a systemd service:
[Service]
OOMScoreAdjust=-500

# At runtime, by PID:
echo -500 > /proc/$(pgrep -f postgres)/oom_score_adj
cat /proc/$(pgrep -f postgres)/oom_score   # observe the effect

Do not use -1000. That makes the process unkillable, and if it is the thing eating memory the kernel will instead kill something you care about even more, or wedge entirely. Reserve -1000 for nothing on a general-purpose box; -500 to -900 is the useful range. Symmetrically, add a positive adjustment to batch jobs whose death is cheap:

[Service]
OOMScoreAdjust=500

Controlling overcommit

vm.overcommit_memory = 0     # heuristic (default)
vm.overcommit_ratio = 50
vm.min_free_kbytes = 65536

Leave overcommit_memory=0 on a general VPS. Setting it to 1 (“always overcommit”) is a copy-paste favourite for Redis, but unless you actually need it for forking under a large dataset, it just removes the kernel’s early sanity check and makes the eventual OOM sharper. Raising min_free_kbytes reserves a small safety margin that keeps reclaim steady instead of letting the kernel reach a cliff.

Using cgroups v2 to bound the damage

The real fix is not letting any single service be able to eat the whole box. With cgroups v2, memory limits are enforced per cgroup, and an over-limit cgroup is reclaimed, then OOM-killed within itself — sparing everything else:

[Service]
MemoryHigh=800M
MemoryMax=1G
MemorySwapMax=256M
MemoryAccounting=yes

MemoryHigh is a soft throttle — the kernel reclaims aggressively and applies pressure to the cgroup, which the application can notice and back off. MemoryMax is the hard wall. Setting MemoryHigh a little below MemoryMax gives the workload a chance to shed memory before it is killed. Verify enforcement with:

systemctl show myapp.service -p MemoryCurrent -p MemoryMax
cat /sys/fs/cgroup/system.slice/myapp.service/memory.events

The memory.events file counts high, max, and oom hits — this is how you know whether your limits are doing anything or are set too generously to ever bind.

Configuring systemd-oomd

[OOM]
SwapUsedLimit=90%
DefaultMemoryPressureLimit=60%
DefaultMemoryPressureDurationSec=20s
ManagedOOMSwap=auto
ManagedOOMMemoryPressure=auto

DefaultMemoryPressureLimit=60% with a 20-second duration means oomd kills a cgroup only after it has sustained 60% pressure for 20 seconds — long enough to avoid killing things during transient spikes.

Per-unit overrides on the units you care about are more important than global settings:

# Protect the database
[Service]
ManagedOOMMemoryPressure=kill
ManagedOOMSwap=kill

# Protect a service entirely from oomd
[Service]
ManagedOOMMemoryPressure=avoid
ManagedOOMSwap=avoid

This is the key insight: avoid and kill control who oomd picks. Put avoid on your database and web tier, and let oomd target the containers or batch jobs that can safely die. Without this, oomd picks the cgroup with the highest pressure, which may well be the database that is merely a victim of another process thrashing the page cache.

Avoiding the oomd/KSM/swap interaction trap

If you have disabled swap entirely, oomd’s swap-limiting heuristics have nothing to act on and it relies purely on PSI, which is fine — but the kernel’s own reclaim then has no escape valve, so latency during pressure gets much worse before the kill. A small swap file (even 1-2 GB on NVMe, with low vm.swappiness) gives reclaim somewhere to put cold pages and makes the whole system far more predictable. Verify that oomd is not fighting a low vm.swappiness by watching /proc/pressure/memory after the change — if full avg10 falls, the combination is working.

Testing that your configuration works

Do not wait for a production OOM to learn whether your tuning is right. Force memory pressure deliberately with a controlled allocator and watch which process dies:

# In a temporary cgroup, allocate until killed
systemd-run --scope -p MemoryMax=200M \
  python3 -c "a=[]
while True: a.append(bytearray(10*1024*1024))"
journalctl -k --since "-2min" | grep -i oom
dmesg | grep -i 'killed process'

The allocation should be killed inside its own cgroup and nothing else should die. If your database gets killed instead, the oom_score_adj or oomd avoid setting is not applied — re-check that the unit was reloaded with systemctl daemon-reload and the service restarted, because these values are read at process start, not on every access.

Monitoring before it fires

Alert on memory.psi.full.avg10 in node-exporter rather than on the OOM event — by the time the OOM message appears you have already lost the service. A threshold alert on sustained full pressure above 5% gives you minutes of warning to investigate, which is usually enough to fix the leak that is causing it.