Transparent Huge Pages on a VPS: When madvise Beats always and Neither Beats Off

Transparent Huge Pages were supposed to be free performance: fewer TLB entries, fewer page faults, less kernel time per byte of memory touched. On a dedicated host with 256 GB of RAM and a settle-down workload they often are. On a 4 GB VPS running a pile of small processes, the same code path spends measurable CPU inside khugepaged scanning and compacting memory that will never form a 2 MB page anyway — and it does that work while holding locks your allocator wants.

The result is a latency tail nobody attributes correctly. Median response time is unchanged, p99 grows by 20-40 ms in bursts every few seconds, and nothing in the application changed. This covers the three THP modes, what khugepaged actually costs, how to measure it from /sys/kernel/mm/transparent_hugepage and /proc/vmstat, and which mode fits which workload. Plan memory size decides whether THP helps at all, so check the provider comparison list against your working set before tuning.

The Three Modes and What Each Promises

always means the kernel attempts huge pages on every anonymous mapping, including the small heap of a 50 MB process. It also enables khugepaged, a background thread that scans for 512 adjacent 4 KiB pages to collapse into one 2 MB page, allocating and copying when it can. madvise disables the automatic attempt: only memory marked MADV_HUGEPAGE gets collapsed, and khugepaged only visits those ranges. never turns off both the fault-time attempt and background collapsing, leaving hugetlbfs as the only huge page mechanism. On a VPS, never is frequently right — not because huge pages are bad, but because collapse requires a copy, and the copy is where the tail latency lives.

cat /sys/kernel/mm/transparent_hugepage/enabled
cat /sys/kernel/mm/transparent_hugepage/defrag
cat /sys/kernel/mm/transparent_hugepage/khugepaged/pages_to_scan
cat /sys/kernel/mm/transparent_hugepage/khugepaged/scan_sleep_millisecs

What khugepaged Costs a Small Guest

khugepaged wakes every 10 seconds, scans up to 4,096 pages for collapsible ranges, and for each candidate takes the range lock, allocates a 2 MB page, copies 512 pages into it, and remaps. The copy is the expensive part: 2 MB of memcpy per collapse plus page table work, under a lock that faults into the same range must wait on.

On a 1 GB guest with a few PHP workers, almost nothing collapses — there are not 512 contiguous aligned pages in active use — but the scan still runs. On a 4 GB guest with a large heap the copies do happen, which is why the same default behaves well on one shape and badly on the other.

Setting	Default	Small VPS (1-4 GB)	Effect
enabled	always	madvise or never	removes background scanning
defrag	madvise	never	no synchronous compaction stalls
pages_to_scan	4096	1024	shorter scan bursts
scan_sleep_millisecs	10000	30000	less frequent wakeups
alloc_sleep_millisecs	60000	60000	unchanged

Compaction Stalls Are the Real Problem

With defrag set to always, an allocation that cannot be satisfied from the free lists triggers direct compaction in the allocating thread. That thread walks memory, migrates pages, and blocks until it either succeeds or gives up after a retry limit. The stall is not the migration itself but the page lock contention it creates for every process touching the pages being moved.

This is why always plus defrag=always is the worst combination on a small guest: it combines background scanning with synchronous stalls in the fault path. If you must use always, set defrag to madvise at minimum, and accept that the fault path will simply fall back to 4 KiB pages instead of blocking.

Measuring It Instead of Guessing

The counters you need are all in /proc/vmstat. thp_fault_alloc counts fault-time allocations, thp_collapse_alloc counts background collapses, and thp_split_page counts pages broken apart again. Two splits per collapse means the collapsing is wasted work. Compare over the same 10-minute window as your latency data.

grep -E 'thp_(fault_alloc|collapse_alloc|split_page|zero)' /proc/vmstat
grep -E 'compact_(stall|success|fail)' /proc/vmstat

# stall time in ms is the last field of /proc/vmstat's compact_stall,
# pair it with latency: run both for 10 minutes, then compare
awk '/thp_collapse_alloc/ {print "collapses", $2}' /proc/vmstat
awk '/thp_split_page/ {print "splits", $2}' /proc/vmstat

# per-process view of anonymous huge pages
grep -A 3 AnonHugePages /proc/meminfo
for p in /proc/[0-9]*/smaps; do grep -l 'AnonHugePages: *[1-9]' $p; done 2>/dev/null | head

A clean signature of the problem: thp_collapse_alloc climbing steadily while thp_split_page climbs at a similar rate, and compact_stall flat. That means background collapsing is churning without producing a stable benefit — the pages are formed and then unmapped or split before they pay back their copy cost.

How Redis, Postgres, and the JVM Respond

Redis is the clearest case for leaving THP off. The allocator uses fork() for background saves, and copy-on-write over 2 MB pages means a child that writes one key dirty an entire 2 MB region. RSS after the fork doubles, and a save that should touch 10 MB of dirty pages can touch hundreds. The Redis documentation has said this for years; it is not an opinion.

PostgreSQL benefits in the shared buffer pool, which is contiguous and long-lived, so madvise plus setting huge_pages = try gets you large pages where it matters. The JVM is the one case where always genuinely helps a large heap, because a 4 GB heap has hundreds of thousands of TLB entries’ worth of references; below 1 GB heap, the win is noise.

Workload	Mode	Reason
Redis / Valkey	never	fork COW amplification
Postgres, pool > 1 GB	madvise	contiguous shmem, no background churn
JVM heap > 2 GB	always, defrag=madvise	TLB reach on a large live set
JVM heap < 1 GB	madvise	no measurable win
PHP-FPM / Node workers	never	short-lived, small allocations
Large in-memory cache, pinned	madvise	collapse once, never split

Making the Change Without a Reboot

enabled and defrag are writable at runtime, and the change affects new mappings only — existing pages keep their current size until they are unmapped. For Redis, set THP to never and then restart Redis so its heap is faulted in fresh. Setting it and leaving the process running accomplishes very little, which is why people conclude the setting does not matter.

echo madvise > /sys/kernel/mm/transparent_hugepage/enabled
echo madvise > /sys/kernel/mm/transparent_hugepage/defrag
# persist without systemd (VPS without unit-file control)
printf '#!/bin/sh\necho madvise > /sys/kernel/mm/transparent_hugepage/enabled\n' \
  > /etc/rc.local && chmod +x /etc/rc.local

A Test Protocol That Produces a Number

Fix the workload, run it for ten minutes under each mode, and record p50, p99, and p99.9 from the application’s own timer — not a proxy, which adds its own queueing. Sample /proc/vmstat deltas alongside. Three runs per mode, because THP effects are bursty and one run lands on either side of the noise.

Expect the difference only in the tail. If p99 moves and p50 does not, the mechanism is compaction or khugepaged contention, and madvise will fix it. If both move, you changed something else too — usually because the restart emptied a cache that had nothing to do with page size. Run it on real memory pressure; the VPS feature breakdown shows the RAM each tier delivers, and THP choices only matter where the working set is large.