{"id":1460,"date":"2026-10-21T22:33:32","date_gmt":"2026-10-21T22:33:32","guid":{"rendered":"https:\/\/virtualserversvps.com\/blog\/?p=1460"},"modified":"2026-10-21T22:33:32","modified_gmt":"2026-10-21T22:33:32","slug":"jemalloc-arena-decay-tuning-redis-nginx-vps","status":"publish","type":"post","link":"https:\/\/virtualserversvps.com\/blog\/jemalloc-arena-decay-tuning-redis-nginx-vps\/","title":{"rendered":"jemalloc Arena and Decay Tuning for Redis and nginx on Memory-Constrained Instances"},"content":{"rendered":"\n

Long-running daemons on small instances accumulate memory that the allocator never returns to the kernel. The process RSS climbs for days while the application’s own live heap stays flat, and eventually the OOM killer picks off whichever process has the highest oom_score<\/code>. The usual cause is glibc’s default arena behaviour: as worker threads allocate, each acquires its own arena, and freed memory fragments into pages that are no longer contiguous enough to return via malloc_trim<\/code>. Linking jemalloc and tuning its decay parameters is the standard fix, and it is measurable in both RSS and p99 latency.<\/p>\n\n\n\n

This walkthrough uses Redis and nginx as the two reference cases because both are multithreaded and both are commonly co-located on the same premium cloud solutions instance<\/a>. The method applies to any C or C++ service you build yourself.<\/p>\n\n\n\n

Confirm that the allocator is actually the problem<\/h2>\n\n\n\n

Start with RSS trend rather than a single snapshot. Pull VmRSS<\/code> every minute for an hour alongside the application’s internal heap metric (Redis INFO memory<\/code> → used_memory<\/code>, or your runtime’s equivalent):<\/p>\n\n\n\n

while true; do ts=$(date +%s); rss=$(grep VmRSS \/proc\/$(pgrep -o redis-server)\/status | awk '{print $2}'); used=$(redis-cli info memory | awk -F: '\/^used_memory:\/{print $2}' | tr -d '\\r'); echo \"$ts $rss $used\"; sleep 60; done<\/code><\/p>\n\n\n\n

If RSS grows by tens of megabytes per day while used_memory<\/code> is flat within a few percent, allocator retention is the explanation. Confirm with malloc_stats()<\/code> semantics by inspecting MALLOC_ARENA_MAX<\/code> behaviour — if there are many arenas and a large RSS-to-live ratio, stop looking elsewhere.<\/p>\n\n\n\n

Signal<\/th>Allocator retention<\/th>Real leak<\/th><\/tr><\/thead>
RSS trend<\/td>Steps up during load, partially returns<\/td>Monotonic, never returns<\/td><\/tr>
Live heap metric<\/td>Flat<\/td>Rising<\/td><\/tr>
After load stops<\/td>Plateau, sometimes falls<\/td>Keeps climbing<\/td><\/tr>
Fragmentation ratio<\/td>mem_fragmentation_ratio<\/code> > 1.5<\/td>Ratio near 1.0 but used_memory rising<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

Install and link jemalloc<\/h2>\n\n\n\n

On Debian and Ubuntu the package is libjemalloc2<\/code> and the shared object lands at \/usr\/lib\/x86_64-linux-gnu\/libjemalloc.so.2<\/code>. Verify before touching anything:<\/p>\n\n\n\n

apt install -y libjemalloc2
ls -l \/usr\/lib\/x86_64-linux-gnu\/libjemalloc.so.2
ldd $(which redis-server) | grep -i jemalloc<\/code><\/p>\n\n\n\n

Redis ships with jemalloc linked by default in official builds; if ldd<\/code> shows it, you are already using the right allocator and the problem is configuration, not linking. For a custom binary or a C service of your own, preload the library instead of rebuilding:<\/p>\n\n\n\n

LD_PRELOAD=\/usr\/lib\/x86_64-linux-gnu\/libjemalloc.so.2 \/usr\/local\/bin\/myapp<\/code><\/p>\n\n\n\n

For a systemd unit, set it in the unit rather than in a shell wrapper so it survives restarts:<\/p>\n\n\n\n

[Service]
Environment=\"LD_PRELOAD=\/usr\/lib\/x86_64-linux-gnu\/libjemalloc.so.2\"
Environment=\"MALLOC_CONF=dirty_decay_ms:5000,muzzy_decay_ms:1000,background_thread:true\"<\/code><\/p>\n\n\n\n

Run systemctl daemon-reload && systemctl restart redis-server<\/code> and check cat \/proc\/$(pgrep -o redis-server)\/maps | grep jemalloc<\/code>. If the library is not mapped, the preload failed silently — systemd strips LD_*<\/code> variables for setuid binaries, so this only works for ordinary services.<\/p>\n\n\n\n

Understand the decay parameters before choosing values<\/h2>\n\n\n\n

jemalloc returns memory in two stages: dirty<\/em> pages are freed to the OS periodically, while muzzy<\/em> pages are reclaimed lazily. Long decay windows improve throughput by reusing hot pages, but let RSS stay high after bursts. Short windows lower RSS but force more madvise<\/code> calls.<\/p>\n\n\n\n