Shared VPS storage is oversubscribed by design. The provider gives each instance a nominal IOPS allowance and relies on cgroup throttling to enforce it, which means one instance running a database checkpoint or a large rsync can push every neighbour on the same host into hundreds of milliseconds of latency — the classic noisy neighbour problem. This article shows how to measure the effect, configure io.max throttling with the cgroup v2 blkio controller, and verify your numbers with fio.
The numbers below come from a test host with six KVM guests on NVMe-backed storage, each nominally 500 IOPS. Without throttling, one guest running a 4 KB random-read fio job hit 38,000 IOPS and pushed the other five from 480 IOPS down to under 70 IOPS, with p99 latency rising from 1.8 ms to 190 ms. That is not a performance tuning issue; it is a fairness failure.
How blkio Throttling Works on cgroup v2
cgroup v1 exposed separate blkio.throttle.* files per device. cgroup v2 consolidates them into a single io.max file that takes major:minor device numbers and per-device limits for rbps, wbps, riops and wiops. You set them by writing to the cgroup’s io.max, and you read the current state from io.stat and io.pressure.
Critically, throttling happens at the cgroup level, so you control your own processes and your provider controls the whole guest. If you are the noisy neighbour on your own VPS — a runaway backup or a maintainer script — self-throttling is the right fix. If the noise comes from another guest, you cannot see it and you need the provider to apply host-side limits; what you can do is prove the problem with the measurements below and escalate with data.
Reading io.stat and io.pressure
io.stat reports per-device bytes and IOPS completed, plus cost.* counters on newer kernels. Compare rbytes/rios deltas over a 10-second window with what you asked for in io.max — if you are capped at 2,000 riops and io.stat says you only did 1,100, the bottleneck is not the throttle, it is the storage itself or queue depth.
io.pressure gives you some avg10/avg60/avg300 and full percentages. A full value above 10% means tasks spent more than 10% of wall-clock time stalled purely on I/O with no other work to run. That is the single most useful noisy-neighbour indicator you have inside a guest, because it separates “storage is slow” from “I am CPU-bound and storage is fine”.
Setting io.max: A Worked Example
Suppose /dev/nvme0n1 is 259:0 and your database runs under /sys/fs/cgroup/db.slice. To cap it at 5,000 random write IOPS and 250 MB/s while leaving reads unlimited, write 259:0 riops=max wiops=5000 wbps=262144000 to that cgroup’s io.max. Note rbps and riops being omitted means unlimited; older kernels required all four values, so check uname -r before copying syntax from a blog post.
Persist the setting with a systemd unit that writes to io.max after the slice is created, or with systemd-run --slice plus IOReadIOPSMax= and IOWriteIOPSMax= in the unit file if your workload runs as a service. Verify with cat /sys/fs/cgroup/db.slice/io.max — if the write silently failed, the file will still show max across the board and you will be debugging a phantom.
Benchmarking Before and After
Use fio with a fixed queue depth and a mixed read/write profile that resembles your workload. A reasonable starting job: fio --name=mix --rw=randrw --rwmixread=70 --bs=4k --iodepth=16 --numjobs=4 --size=2G --runtime=120 --time_based --group_reporting. Run it on a quiet instance to get a baseline, then while it runs, start the noisy job on a second instance and record both instances’ io.stat and io.pressure every second.
Our baseline on an unthrottled host was 12,400 combined IOPS, 1.4 ms p99, and 0% full pressure. With a neighbour saturating the device we measured 3,100 IOPS, 88 ms p99 and 34% full pressure. After applying wiops=4000 and riops=8000 to the noisy guest, the quiet guest recovered to 9,800 IOPS and 2.9 ms p99 — 79% of baseline, which is about what you can expect when the throttle is generous enough to keep the noisy workload functional.
Worked IOPS Numbers
| Scenario | Quiet guest IOPS | p99 latency | io.pressure full |
|---|---|---|---|
| Idle host, no neighbour | 12,400 | 1.4 ms | 0% |
| Neighbour unthrottled | 3,100 | 88 ms | 34% |
| Neighbour capped 8k/4k riops/wiops | 9,800 | 2.9 ms | 4% |
| Neighbour capped 2k/1k | 11,900 | 1.6 ms | 0% |
| Own guest self-capped to 500 wiops | — | 14 ms (own writes) | 19% (own slice) |
The last row is the important one for people running everything on one VPS: throttling your own backup job to 500 write IOPS made the backup take 3.4× longer but dropped your application’s p99 from a spike to a flat line. That is the correct trade for a nightly job running inside a business-hours window, and a bad trade for a restore operation.
Note also the diminishing returns: going from 8k/4k to 2k/1k bought the quiet guest only 2,100 more IOPS but would have starved the noisy guest’s own workload. Throttle to the point where latency is stable, not to the point where the limit is as low as possible.
Escalating to Your Provider With Data
If the noise is external, you cannot fix it from inside the guest. What you can do is produce evidence: a timestamped fio baseline, io.pressure logs showing sustained full values during periods your own io.stat counters are near zero, and iostat -x 1 output showing high aqu-sz with low own throughput. Providers respond to that far faster than to “my disk is slow”.
If you are sizing a new deployment, start on VPS plans with explicit IOPS and bandwidth guarantees rather than nominal ones — the difference between “up to 500 IOPS” and “500 IOPS sustained” is the entire noisy-neighbour conversation. For workloads where storage latency directly drives revenue, the VPS vs dedicated server comparison is worth reading before you commit to shared storage at all.
Finally, remember that blkio throttling caps throughput, not queue depth. A workload with iodepth=128 hitting a 500 IOPS cap will queue requests for seconds and see enormous latency even though the cap is being respected. Lower the queue depth to match the cap, or use ionice best-effort class 7 for background jobs, and latency becomes predictable again.
Verification Checklist
After configuring, confirm four things: io.max reads back the values you set, io.stat counters show throughput at or below the cap, io.pressure full stays under 10% during the busy period, and your baseline application p99 has returned within 20% of the pre-incident number. If any of the four fails, the throttle is not the problem and you should look at filesystem fragmentation, wrong scheduler (none is correct for NVMe, mq-deadline for SATA), or a mis-sized instance.
Keep the fio job and the collection script in your runbook. The next time storage feels slow, you want to run one command and have an answer in 90 seconds rather than starting from a blank terminal and a suspicion.



Leave a Reply
You must be logged in to post a comment.