{"id":1646,"date":"2026-11-08T22:41:50","date_gmt":"2026-11-08T22:41:50","guid":{"rendered":"https:\/\/virtualserversvps.com\/blog\/?p=1646"},"modified":"2026-11-08T22:41:50","modified_gmt":"2026-11-08T22:41:50","slug":"journald-storage-flush-rate-log-storm","status":"publish","type":"post","link":"https:\/\/virtualserversvps.com\/blog\/journald-storage-flush-rate-log-storm\/","title":{"rendered":"Journald Storage Caps and Flush Rate During a Log Storm"},"content":{"rendered":"
\n

When a service starts failing rapidly, the flood of log lines is the diagnostic evidence you need and simultaneously the thing that fills the disk. Journald has two separate mechanisms that decide what happens next: storage caps that limit total bytes, and a rate limiter that decides how fast entries are accepted. Understanding both is the difference between keeping the evidence and corrupting the journal mid-incident.<\/p>\n

The behaviour changes between systemd versions and depends on whether the journal is volatile, persistent, or forwarded to a remote collector. This article covers configuration, the rate limiter’s internals, and the measurement approach for sizing caps against real log volume.<\/p>\n

Where Journald Puts Data<\/h2>\n

On most distributions the journal is at `\/var\/log\/journal`, which makes it persistent across reboots. If that directory does not exist, the journal is volatile and lives under `\/run\/log\/journal`, which sits in RAM and is lost on restart. Check with `journalctl –header | grep -i ‘^File’` to see the actual file path in use.<\/p>\n

Persistent journals survive a crash, which is exactly when you want the log; on a small instance the VPS hosting overview<\/a> lists plans where the root disk has room for several gigabytes of history. Enable persistence with `mkdir -p \/var\/log\/journal` and `systemd-tmpfiles –create –prefix \/var\/log\/journal`, then restart journald. Note that this doubles memory use while the journal is being written, because journald keeps a write cache and copies to disk.<\/p>\n

The Storage Caps<\/h2>\n

`SystemMaxUse` caps total disk usage and is the directive people actually set. `SystemKeepFree` reserves free space so the journal never fills the disk. `SystemMaxFileSize` limits individual journal files, and `SystemMaxFiles` limits the count. Defaults are 10% of the filesystem and 15% free space, which on a 40 GB VPS means a 4 GB cap and a 6 GB reserve.<\/p>\n

Set values that suit the workload. A busy web server producing 2 GB of logs per day with a 4 GB cap keeps only two days of history, which is not enough to investigate a problem discovered on Monday for an event that happened Friday. Raise the cap to 15 GB or forward to a remote collector. Test after each change with `journalctl –disk-usage`.<\/p>\n

The Rate Limiter and Where It Bites<\/h2>\n

Journald applies rate limiting per service with `RateLimitIntervalSec` (default 30 s) and `RateLimitBurst` (default 10000). Exceeding the burst within the interval causes entries to be dropped, and journald logs a single summary line saying how many were suppressed. Those suppressed lines are exactly the ones a log storm produces.<\/p>\n

The limiter is per-service, not global, so one noisy unit does not starve the others. That is a good design until you are debugging that noisy unit. Set `RateLimitBurst=0` for the specific service in its unit file via `LogRateLimitBurst=` to disable the limit for that unit, and keep the global limit as a safety valve.<\/p>\n

Measuring Volume Before You Set Caps<\/h2>\n

Do not guess at growth. Measure what the machine actually writes over a representative window. `journalctl –since ’24 hours ago’ -o short-iso | wc -c` gives total bytes, and `journalctl –since ’24 hours ago’ -o json | jq -r ‘.SYSLOG_IDENTIFIER’ | sort | uniq -c | sort -rn` gives the per-unit breakdown.<\/p>\n

The breakdown is what matters. A single unit producing 80% of the bytes usually has a bug or a misconfigured debug level. Fix that first; storage caps are a last resort and hide the real problem. `du -sh \/var\/log\/journal\/*\/` shows the size per directory and per user journal.<\/p>\n

Flush Rate and Disk Pressure<\/h2>\n

Journald writes to disk asynchronously. Under a storm, the write cache grows and flushes in bursts, which can saturate a shared VPS disk and slow everything else. `vmstat 1` during the event shows the `bo` column; sustained values above a few thousand blocks per second on a small VPS coincide with application latency.<\/p>\n

Reduce the flush pressure by raising `RuntimeMaxUse` so more of the journal stays in memory, or by sending verbose units to a remote syslog receiver with `ForwardToSyslog=yes`. Check `journalctl –verify` after a storm; a full disk during a flush can leave journal files with a corrupted tail, which journald marks and quarantines.<\/p>\n

Protecting the Disk From the Storm<\/h2>\n

Set `SystemKeepFree` to at least 10% of the disk so the journal stops writing before the filesystem fills. The kernel needs free space for ext4 metadata and for temporary files during package operations; a full `\/var` breaks far more than logging. Verify with `df -h \/var` that the reserve is real, not assumed.<\/p>\n

Separate `\/var` or `\/var\/log` onto its own volume when the workload is bursting. A log storm that fills the journal volume then leaves the application volume untouched, which keeps the site serving while the evidence is preserved. This is the one change that converts a log storm from an outage into an inconvenience.<\/p>\n

Forwarding as the Real Answer<\/h2>\n

Local caps are a budget, not an archive. Forward entries to a collector outside the instance so the amount of history is decoupled from disk size. `journalctl -o json –follow` piped to a shipper, or `ForwardToSyslog=yes` with rsyslog to a remote host, both work; the JSON format preserves structured fields that plain text loses.<\/p>\n

Set the collector’s retention independently, and alarm on ingestion rate so a storm is visible in the collector as a volume spike rather than a disk full incident on production. Combine forwarding with a modest local cap and you get both short-term forensics and long-term history. For the instance side, see our VPS plan breakdown<\/a> when choosing a disk size that leaves room for a separate log volume.<\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

When a service starts failing rapidly, the flood of log lines is the diagnostic evidence you need and simultaneously the thing that fills the disk. Journald has two separate mechanisms…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"iawp_total_views":1,"footnotes":""},"categories":[3],"tags":[],"class_list":["post-1646","post","type-post","status-publish","format-standard","hentry","category-performance-optimization"],"yoast_head":"\nJournald Storage Caps and Flush Rate During a Log Storm - Virtual Servers VPS Blog<\/title>\n<meta name=\"description\" content=\"When a service starts failing rapidly, the flood of log lines is the diagnostic evidence you need and simultaneously the thing that fills the disk.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/virtualserversvps.com\/blog\/journald-storage-flush-rate-log-storm\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Journald Storage Caps and Flush Rate During a Log Storm\" \/>\n<meta property=\"og:description\" content=\"Journald Storage Caps and Flush Rate During a Log Storm\" \/>\n<meta property=\"og:url\" content=\"https:\/\/virtualserversvps.com\/blog\/journald-storage-flush-rate-log-storm\/\" \/>\n<meta property=\"og:site_name\" content=\"Virtual Servers VPS Blog\" \/>\n<meta property=\"article:published_time\" content=\"2026-11-08T22:41:50+00:00\" \/>\n<meta name=\"author\" content=\"Virtual-Servers-Vps-Editor\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Virtual-Servers-Vps-Editor\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/virtualserversvps.com\/blog\/journald-storage-flush-rate-log-storm\/\",\"url\":\"https:\/\/virtualserversvps.com\/blog\/journald-storage-flush-rate-log-storm\/\",\"name\":\"Journald Storage Caps and Flush Rate During a Log Storm - Virtual Servers VPS Blog\",\"isPartOf\":{\"@id\":\"https:\/\/virtualserversvps.com\/blog\/#website\"},\"datePublished\":\"2026-11-08T22:41:50+00:00\",\"author\":{\"@id\":\"https:\/\/virtualserversvps.com\/blog\/#\/schema\/person\/82a299a8284a66ff49f97c74684724a0\"},\"description\":\"When a service starts failing rapidly, the flood of log lines is the diagnostic evidence you need and simultaneously the thing that fills the disk.\",\"breadcrumb\":{\"@id\":\"https:\/\/virtualserversvps.com\/blog\/journald-storage-flush-rate-log-storm\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/virtualserversvps.com\/blog\/journald-storage-flush-rate-log-storm\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/virtualserversvps.com\/blog\/journald-storage-flush-rate-log-storm\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/virtualserversvps.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Journald Storage Caps and Flush Rate During a Log Storm\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/virtualserversvps.com\/blog\/#website\",\"url\":\"https:\/\/virtualserversvps.com\/blog\/\",\"name\":\"Virtual Servers VPS Blog\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/virtualserversvps.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/virtualserversvps.com\/blog\/#\/schema\/person\/82a299a8284a66ff49f97c74684724a0\",\"name\":\"Virtual-Servers-Vps-Editor\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/virtualserversvps.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/d820b15f1cd028e97610d9adf536df7be5cb6423869967037d468d5355fa003f?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/d820b15f1cd028e97610d9adf536df7be5cb6423869967037d468d5355fa003f?s=96&d=mm&r=g\",\"caption\":\"Virtual-Servers-Vps-Editor\"},\"sameAs\":[\"https:\/\/virtualserversvps.com\/blog\"],\"url\":\"https:\/\/virtualserversvps.com\/blog\/author\/virtualserversvps\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Journald Storage Caps and Flush Rate During a Log Storm - Virtual Servers VPS Blog","description":"When a service starts failing rapidly, the flood of log lines is the diagnostic evidence you need and simultaneously the thing that fills the disk.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/virtualserversvps.com\/blog\/journald-storage-flush-rate-log-storm\/","og_locale":"en_US","og_type":"article","og_title":"Journald Storage Caps and Flush Rate During a Log Storm","og_description":"Journald Storage Caps and Flush Rate During a Log Storm","og_url":"https:\/\/virtualserversvps.com\/blog\/journald-storage-flush-rate-log-storm\/","og_site_name":"Virtual Servers VPS Blog","article_published_time":"2026-11-08T22:41:50+00:00","author":"Virtual-Servers-Vps-Editor","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Virtual-Servers-Vps-Editor","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/virtualserversvps.com\/blog\/journald-storage-flush-rate-log-storm\/","url":"https:\/\/virtualserversvps.com\/blog\/journald-storage-flush-rate-log-storm\/","name":"Journald Storage Caps and Flush Rate During a Log Storm - Virtual Servers VPS Blog","isPartOf":{"@id":"https:\/\/virtualserversvps.com\/blog\/#website"},"datePublished":"2026-11-08T22:41:50+00:00","author":{"@id":"https:\/\/virtualserversvps.com\/blog\/#\/schema\/person\/82a299a8284a66ff49f97c74684724a0"},"description":"When a service starts failing rapidly, the flood of log lines is the diagnostic evidence you need and simultaneously the thing that fills the disk.","breadcrumb":{"@id":"https:\/\/virtualserversvps.com\/blog\/journald-storage-flush-rate-log-storm\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/virtualserversvps.com\/blog\/journald-storage-flush-rate-log-storm\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/virtualserversvps.com\/blog\/journald-storage-flush-rate-log-storm\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/virtualserversvps.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Journald Storage Caps and Flush Rate During a Log Storm"}]},{"@type":"WebSite","@id":"https:\/\/virtualserversvps.com\/blog\/#website","url":"https:\/\/virtualserversvps.com\/blog\/","name":"Virtual Servers VPS Blog","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/virtualserversvps.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/virtualserversvps.com\/blog\/#\/schema\/person\/82a299a8284a66ff49f97c74684724a0","name":"Virtual-Servers-Vps-Editor","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/virtualserversvps.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/d820b15f1cd028e97610d9adf536df7be5cb6423869967037d468d5355fa003f?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/d820b15f1cd028e97610d9adf536df7be5cb6423869967037d468d5355fa003f?s=96&d=mm&r=g","caption":"Virtual-Servers-Vps-Editor"},"sameAs":["https:\/\/virtualserversvps.com\/blog"],"url":"https:\/\/virtualserversvps.com\/blog\/author\/virtualserversvps\/"}]}},"_links":{"self":[{"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/posts\/1646","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/comments?post=1646"}],"version-history":[{"count":1,"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/posts\/1646\/revisions"}],"predecessor-version":[{"id":1647,"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/posts\/1646\/revisions\/1647"}],"wp:attachment":[{"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/media?parent=1646"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/categories?post=1646"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/tags?post=1646"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}