Log Rotation on a VPS: Compression, Retention Windows and Reclaiming Disk Before You Panic

A full disk is the most common avoidable outage on a small VPS, and logs are almost always the culprit. The default logrotate configuration ships with retention tuned for generous enterprise disks, not for a 25 GB instance running nginx, PHP-FPM and a database. This article shows how to compress intelligently, set retention by both count and total size, and verify that rotation is actually reclaiming space instead of silently failing.

Find out what is actually consuming disk

Before changing policy, establish where the space went. Log directories are frequently larger than owners expect, and a single runaway file often dominates.

du -xh --max-depth=2 /var/log | sort -h | tail -15

# ten largest files anywhere under /var
find /var/log -type f -printf '%s\t%p\n' | sort -n | tail -10 | numfmt --field=1 --to=iec

# is anything still holding a deleted file open?
lsof +L1 2>/dev/null | awk '$7 > 100000000 {print $7, $9}'

That last command matters: deleting a log file a process still has open frees nothing until the process restarts. Rotation must signal the process, which logrotate does via postrotate.

A sane baseline policy

The global file sets defaults; per-service overrides refine them. Compression should be the default, with delayed compression so the freshly rotated file stays greppable for a short while.

# /etc/logrotate.d/nginx
/var/log/nginx/*.log {
    daily
    rotate 14
    missingok
    notifempty
    compress
    delaycompress
    compresscmd /usr/bin/zstd
    uncompresscmd /usr/bin/unzstd
    compressext .zst
    dateext
    dateformat -%Y%m%d
    sharedscripts
    create 0640 www-data adm
    postrotate
        [ -f /run/nginx.pid ] && kill -USR1 $(cat /run/nginx.pid)
    endscript
}

dateext prevents the numeric-shuffle churn and makes retention auditing trivial. delaycompress keeps yesterday’s file plain-text so ordinary tooling still works during incidents.

Compression: zstd vs gzip vs xz

Codec	Ratio	Speed	CPU per rotation
gzip -6	~10x	medium	moderate
zstd -3	~10x	very fast	low
zstd -19	~14x	slow	high
xz -6	~16x	very slow	very high

On a low-core VPS, zstd at level 3 or 6 is the sweet spot: near-gzip ratios at a fraction of the CPU, so a nightly rotation does not spike load. Reserve xz for archives you compress once and query rarely. Verify a codec is available before switching, or logrotate will fail the whole stanza:

command -v zstd zstdcat xz

# dry-run the new config across all matching logs
logrotate --debug /etc/logrotate.d/nginx

# force a rotation now, outside the cron window
logrotate --force /etc/logrotate.d/nginx

Retention by size, not just by count

rotate N caps the number of archives, but a file that grows tenfold suddenly means ten times the disk. Cap total size too, using maxsize to rotate early.

# rotate early if the file exceeds 200 MB, regardless of count
/var/log/app/*.log {
    daily
    rotate 10
    maxsize 200M
    compress
    copytruncate      # use when the app cannot reopen its log fd
}

copytruncate is convenient but has a race window where a few lines can be lost between copy and truncate; prefer postrotate + signal whenever the application supports reopening its file descriptors.

The journald side of the equation

On systemd hosts, /var/log/journal grows independently of logrotate and is a frequent hidden consumer. Cap it explicitly:

# /etc/systemd/journald.conf
SystemMaxUse=200M
SystemMaxFileSize=50M
MaxRetentionSec=2week

systemctl restart systemd-journald
journalctl --disk-usage
journalctl --vacuum-size=200M

Per-application retention that matches value

Not every log deserves the same lifespan. Access logs are forensic evidence; debug logs from an application are transient noise. Splitting retention by value is how you keep two weeks of the first and two days of the second on the same disk.

# high-value: keep, compress well, never lose
/var/log/nginx/*.log {
    daily
    rotate 30
    compress
    delaycompress
    dateext
}

# noisy debug output: short window, aggressive cap
/var/log/app/debug.log {
    daily
    rotate 3
    maxsize 100M
    compress
    copytruncate
    missingok
    notifempty
}

Two details matter here. notifempty prevents rotation churn on logs that stayed silent. And mixing dateext only on the high-value stanza keeps the debug directory simple while giving the forensic logs stable, sortable filenames.

Rotation timing and the cron window

logrotate is normally invoked by a systemd timer once per day. If you add a fresh /etc/logrotate.d stanza after that window has passed, nothing happens until tomorrow — the single most common reason a new policy appears to be ignored.

# who triggers rotation on modern Debian/Ubuntu
systemctl list-timers logrotate.timer

# inspect what the daily job actually iterates over
grep -v '^#' /etc/logrotate.conf

# test a single stanza without waiting for the timer
logrotate --force --verbose /etc/logrotate.d/nginx

Use --force sparingly in production: it rotates immediately regardless of the size or age conditions, which is exactly what you want while validating a config and exactly what you do not want on a live database host mid-day.

Preventing rotation from failing silently

A misconfigured stanza fails quietly and leaves the previous archives intact, so disk usage climbs with no obvious error. logrotate writes its state to a status file; a healthy run updates the timestamp, and a repeated failure shows a stale entry.

# last successful rotation per log
cat /var/lib/logrotate/status | grep /var/log/nginx

# capture errors from a manual run
logrotate --verbose --debug /etc/logrotate.d/nginx 2>&1 | tail -20

# catch failures in a timer-driven setup
systemctl status logrotate.service --no-pager

• Missing compresscmd binary → whole stanza skipped.
• Bad glob with no missingok → stanza errors and stops.
• postrotate signal sent to a stopped service → old file kept open, space not reclaimed.
• Permissions on the create line wrong for the daemon → app cannot write after the first rotation.

Reclaiming space from archives you will never read

Old compressed archives are the cheapest thing to delete, so build the habit of inspecting before purging. Because dateext encodes the date in the filename, retention becomes a simple text filter.

# how much space is sitting in archives, by month
du -ch /var/log/nginx/*.zst 2>/dev/null | tail -1

# list archives older than 30 days before removing anything
find /var/log -name '*.zst' -mtime +30 -printf '%TY-%Tm-%Td %10s %p\n' | sort

# grep a rotated log without decompressing to disk
zstdcat /var/log/nginx/access.log-20260101.zst | grep ' 500 '

Searching compressed logs in place with zstdcat keeps the convenience of plain text without paying the disk cost of decompression. This is the reason delaycompress exists: the most recent archive stays readable by ordinary tools during the window when you are most likely to need it.

A lightweight disk-capacity guard

Even a perfect policy benefits from an independent watchdog. A tiny script on a timer that alerts when /var crosses a threshold turns a future outage into a routine notification.

#!/bin/sh
# /usr/local/bin/disk-guard.sh
THRESH=80
USE=$(df --output=pcent /var | tail -1 | tr -d ' %')
[ "$USE" -ge "$THRESH" ] && \
  logger -t disk-guard "WARNING: /var at ${USE}%" && \
  du -xh --max-depth=1 /var/log | sort -h | tail -5 | logger -t disk-guard

Summary

• Compress with zstd at level 3-6; delay compression so the newest archive stays greppable.
• Cap both count (rotate) and size (maxsize) — one without the other is a trap.
• Bound systemd-journald separately; it ignores logrotate entirely.
• Alert at 80% of /var rather than discovering the problem from a 500 error.

Pairing a tight rotation policy with a capacity alert is the difference between a boring log directory and a 03:00 outage. If disk pressure persists after both are in place, the instance may simply be undersized — the plan comparison is worth a look, and the blog covers the related disk-reclamation procedures.