Benchmarking NVMe I/O Throughput on a KVM VPS with fio and iostat

“NVMe-backed storage” is a marketing line on nearly every VPS product page. Whether the storage on your specific instance actually delivers NVMe-class throughput is something you can measure in about twenty minutes with two standard tools. This article gives you a reproducible procedure for benchmarking disk I/O on a KVM VPS, interpreting the numbers honestly, and recognising the throttling that shared storage quietly applies. Disk latency is the single biggest driver of web TTFB on a small host, so getting a real baseline matters before you blame the application. If you want the surrounding context first, the fio and dd benchmark methodology on the blog covers the fundamentals.

What NVMe Should Deliver Versus What a VPS Gives You

A bare NVMe SSD can sustain hundreds of thousands of random 4K IOPS and sub-100-microsecond latency. On a KVM VPS you will not see those numbers, and that is expected: your virtual disk is a file or logical volume on the host, going through the virtio-blk or virtio-scsi stack, and sharing physical devices with other tenants. The realistic targets for a well-provisioned VPS are:

• Sequential read: 500–2000 MB/s
• Random 4K read: 5,000–50,000 IOPS depending on plan tier
• Average 4K latency: 200 µs – 2 ms

If your numbers fall far below these, or fluctuate wildly run to run, you are either on oversubscribed storage or hitting an explicit throttle — and the method below will show you which.

Baseline: Confirm the Device and Scheduler

Identify your block device and check the I/O scheduler. Modern NVMe benefits from none (no scheduler); legacy mq-deadline or bfq can add latency under parallel load:

lsblk -d -o NAME,ROTA,SCHED,SIZE
cat /sys/block/vda/queue/scheduler
cat /sys/block/vda/queue/rotational

ROTA=0 confirms the device is presented as non-rotational. If the scheduler shows mq-deadline on a modern kernel, test whether switching to none improves latency before drawing conclusions about the storage itself.

Sequential Throughput with fio

Start with sustained sequential reads and writes to establish the ceiling. Use a 1 GiB file, a queue depth of 32, and direct I/O to bypass the page cache — otherwise you are benchmarking RAM, not disk:

sudo apt install fio -y

# Sequential read
fio --name=seqread --rw=read --bs=128k --direct=1 \
  --size=1G --numjobs=1 --iodepth=32 --runtime=30 \
  --time_based --group_reporting

# Sequential write
fio --name=seqwrite --rw=write --bs=128k --direct=1 \
  --size=1G --numjobs=1 --iodepth=32 --runtime=30 \
  --time_based --group_reporting

Read the bw (bandwidth) line in the output. --direct=1 is non-negotiable for a meaningful result; without it, fio’s writes complete into the kernel page cache and report absurd throughput that says nothing about the backing device.

Random 4K IOPS — The Number That Predicts Real Behaviour

Web and database workloads generate random small-block I/O, not large sequential transfers. This is the test that matters most:

fio --name=randread --rw=randread --bs=4k --direct=1 \
  --size=1G --numjobs=4 --iodepth=32 --runtime=30 \
  --time_based --group_reporting

fio --name=randwrite --rw=randwrite --bs=4k --direct=1 \
  --size=1G --numjobs=4 --iodepth=32 --runtime=30 \
  --time_based --group_reporting

The key output fields are iops and the latency percentiles (clat — completion latency). Watch the 99.00th percentile, not the average. A storage backend with a good average but a 99th-percentile latency of 50 ms will cause visible request stalls in production even though its mean looks healthy. This percentile-focused reading is the same discipline described in interpreting VPS benchmarks correctly.

Corroborating with iostat During the Run

fio reports what the tool observed; iostat reports what the kernel saw at the device level. Running them together catches illusions. In a second terminal during the fio run:

sudo apt install sysstat -y
iostat -xz 1 30

Look at %util (device utilisation) and await (average wait). If %util pins at 100% while fio reports low IOPS, the device is saturated — the bottleneck is the backing store, not your configuration. If await climbs steadily over the run, you are likely hitting a rate limit rather than a hardware ceiling.

Detecting Throttling

Many VPS plans cap IOPS via cgroup v2 io.max or host-level token buckets. Tick-based throttling shows a signature: throughput is rock-steady at a suspiciously round number (e.g. exactly 5,000 IOPS every run) rather than varying with load. Re-run the random-read test three times and compare. Natural storage varies a few percent between runs; a throttle produces an eerily identical figure. To see it from the process side, watch your own I/O pressure while the test runs:

cat /proc/pressure/io

A rising some avg60 value during the benchmark confirms processes are stalling on I/O — the definition of pressure. Understanding how to read PSI is covered in depth in reading Linux PSI to diagnose VPS bottlenecks.

Interpreting the Results Honestly

Run every test three times and report the median, not the best run. A single lucky run overstates capability. Compare sequential versus random: if sequential is fast but random 4K is an order of magnitude worse, the storage is likely a cached tier or network-attached volume rather than local NVMe. If both are weak and 99th-percentile latency is high, the host is oversubscribed and you should test at a different hour to confirm before changing providers.

The Takeaway

Benchmarking NVMe on a KVM VPS comes down to four habits: always use direct I/O, focus on random 4K IOPS and the 99th-percentile latency rather than averages, corroborate fio with iostat at the device level, and run each test three times. That routine distinguishes genuinely fast storage from a marketing claim — and tells you exactly when a disk is the reason your application feels slow.