{"id":1331,"date":"2026-10-07T23:58:07","date_gmt":"2026-10-07T23:58:07","guid":{"rendered":"https:\/\/virtualserversvps.com\/blog\/?p=1331"},"modified":"2026-10-07T23:58:07","modified_gmt":"2026-10-07T23:58:07","slug":"auditing-systemd-journal-disk-usage-small-vps","status":"publish","type":"post","link":"https:\/\/virtualserversvps.com\/blog\/auditing-systemd-journal-disk-usage-small-vps\/","title":{"rendered":"Auditing systemd Journal Disk Usage on a Small VPS: Measure, Cap and Verify"},"content":{"rendered":"
\n\n\n

On a small VPS with a 20\u201340 GB disk, an unmanaged systemd journal is one of the most common causes of a surprise full filesystem. Unlike logrotate-driven text logs, the journal is a binary store with its own retention logic, and its size is not obvious from du<\/code> on \/var\/log<\/code> until you know where to look. This is a measurement-first audit: find out what the journal actually holds, how fast it grows, then cap it deliberately.<\/p>\n\n\n\n

Measure Before You Cap Anything<\/h2>\n\n\n\n

journalctl<\/code> reports its own disk usage directly, which is more reliable than guessing from directory sizes:<\/p>\n\n\n\n

journalctl --disk-usage\ndu -sh \/var\/log\/journal \/run\/log\/journal 2>\/dev\/null\nls -lh \/var\/log\/journal\/*\/ | head<\/code><\/pre>\n\n\n\n
Two locations matter. \/run\/log\/journal<\/code> is a tmpfs store that vanishes on reboot; \/var\/log\/journal<\/code> is persistent. If only the former exists, your logs are not surviving reboots at all \u2014 a separate problem worth fixing if you need post-mortem forensics.<\/p>\n\n\n\n
Find Who Is Flooding the Journal<\/h2>\n\n\n\n
A journal that balloons from a few hundred MB to several GB in a day is usually one misbehaving unit writing in a tight loop. Rank units by volume:<\/p>\n\n\n\n
journalctl --since \"24 hours ago\" --output=json \\\n  | jq -r '.SYSLOG_IDENTIFIER \/\/ ._COMM' | sort | uniq -c | sort -rn | head -20<\/code><\/pre>\n\n\n\n
If jq<\/code> is not installed and you want a lighter view, use journalctl --since \"1 hour ago\" | awk '{print $5}' | sort | uniq -c | sort -rn<\/code>. Either way the goal is to identify the top talker. Common culprits are a failing service crash-looping, an application logging every request at debug level, or a kernel warning repeated thousands of times per minute.<\/p>\n\n\n\n
Cap Retention in journald.conf<\/h2>\n\n\n\n
journald offers three independent limits \u2014 size, time, and free-space floor. Set all three; they interact, and whichever triggers first wins.<\/p>\n\n\n\n
sudo tee \/etc\/systemd\/journald.conf.d\/size.conf <<'EOF'\n[Journal]\nSystemMaxUse=500M\nSystemKeepFree=1G\nMaxRetentionSec=2week\nMaxFileSec=1week\nCompress=yes\nStorage=persistent\nEOF\nsudo systemctl restart systemd-journald<\/code><\/pre>\n\n\n\n
Directive<\/th>Purpose<\/th>Suggested value<\/th><\/tr><\/thead>
SystemMaxUse<\/td>Hard ceiling on journal size<\/td>500M\u20131G<\/td><\/tr>
SystemKeepFree<\/td>Always leave disk free<\/td>1G<\/td><\/tr>
MaxRetentionSec<\/td>Delete entries older than<\/td>2week<\/td><\/tr>
MaxFileSec<\/td>Rotate a journal file after<\/td>1week<\/td><\/tr>
Compress<\/td>Compress rotated files<\/td>yes<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
SystemKeepFree<\/code> is the safety net: even if a log flood occurs, journald stops before the disk fills and takes your services down with it. On a 20 GB VPS, keeping a 1 GB floor is cheap insurance.<\/p>\n\n\n\n
Force Rotation and Reclaim Immediately<\/h2>\n\n\n\n
Config changes apply to new writes; existing data must be rotated and vacuumed explicitly. Do this in the right order:<\/p>\n\n\n\n
sudo journalctl --rotate\nsudo journalctl --vacuum-size=500M\nsudo journalctl --vacuum-time=2weeks\njournalctl --disk-usage<\/code><\/pre>\n\n\n\n
--rotate<\/code> seals the current active file; --vacuum-size<\/code> then deletes oldest entries until the cap is met. Run --disk-usage<\/code> afterward to confirm the reclaim actually happened.<\/p>\n\n\n\n
Set Caps Per-Unit, Not Just Globally<\/h2>\n\n\n\n
One chatty unit can still consume the entire global budget and evict everything else. Limit the noisy service at the source with LogRateLimit*<\/code>:<\/p>\n\n\n\n