How to Benchmark Disk IOPS on a VPS with fio: Queue Depth and Latency That Actually Matter

A 4k random read test that returns 40,000 IOPS on Monday and 12,000 IOPS on Tuesday is not measuring the same thing twice. Queue depth, block size, filesystem cache and the guest’s block layer all change the result, and most benchmark blog posts change them silently between runs. The number that matters for a database or a mail server is not peak IOPS at queue depth 256 with a warm page cache — it is sustained 4k random IOPS at the queue depth your workload actually generates, measured after the cache has been bypassed.

This is a fixed fio protocol that produces comparable numbers across VPS providers. Every test below states what it measures, the fio flags that make it honest, and the threshold that separates a plan with real NVMe from one with a capped virtual disk. The whole sequence takes about 12 minutes on a 2 vCPU instance.

Understand what the disk path actually is

Before running fio, establish three facts about the device: whether it is rotational, which scheduler is attached, and what the provider’s throttle is. A KVM guest on NVMe still presents a virtio-blk or virtio-scsi device, and the host may enforce an IOPS cap below the physical capability of the drive.

lsblk -d -o NAME,ROTA,SCHED,SIZE,MODEL
cat /sys/block/vda/queue/scheduler
cat /sys/block/vda/queue/nr_requests
cat /sys/block/vda/queue/rotational
# physical block size and alignment
cat /sys/block/vda/queue/physical_block_size
cat /sys/block/vda/queue/logical_block_size

ROTA=0 means the guest is told the device is non-rotational. That is a hint, not a guarantee — providers routinely present SATA SSDs and network-backed block storage as non-rotational. nr_requests tells you the queue depth the kernel will use; values of 128 or 256 are typical for virtio-blk, and the host may collapse anything beyond a much smaller internal queue.

Why direct I/O is mandatory for honest results

Without --direct=1, fio writes into the page cache. A 2 GB test file on a box with 4 GB of RAM is served entirely from memory after the first pass, and you measure RAM bandwidth rather than storage. The same applies to reads: the second run of a read test on a cached file will report hundreds of thousands of IOPS because no I/O reached the device.

• --direct=1 — bypass the page cache, force every operation to the device
• --iodepth=32 — queue depth; match your workload, 16–32 is realistic for a database
• --numjobs=4 — concurrency; four jobs at depth 32 gives an effective queue depth of 128
• --runtime=60 --time_based — fixed duration so results are comparable run to run
• --ramp_time=10 — discard the first 10 seconds while the queue fills
• --size=2G per job — large enough to defeat any remaining cache, small enough to finish
• --output-format=json — parse it, do not eyeball the text table

If you are still deciding which plan to rent, the VPS plans listed on the main site document the storage type per tier, which is the single most useful column on any provider comparison page.

The four tests that matter

Run these in order. Each one answers a different question, and a provider that passes the first and fails the second is a common pattern: fast burst, hard throttle.

Test 1: 4k random read, queue depth 32

fio --name=randread4k \
  --filename=/root/fio-test.bin --size=2G --direct=1 \
  --rw=randread --bs=4k --iodepth=32 --numjobs=1 \
  --ioengine=libaio --runtime=60 --time_based --ramp_time=10 \
  --group_reporting --output-format=json > /root/randread4k.json
jq '.jobs[0].read.iops, .jobs[0].read.clat_ns.mean/1000' /root/randread4k.json

Read the IOPS and the mean completion latency together. A plan reporting 45,000 IOPS at 0.7 ms is healthy NVMe. A plan reporting 45,000 IOPS at 14 ms is not doing 45,000 IOPS — it is queueing, and the reported figure is the rate at which a saturated queue drains. Divide IOPS by 1000 and compare to latency in milliseconds: if mean latency in ms is roughly queue depth divided by IOPS in thousands, the device is saturated and the number is a ceiling.

Test 2: 4k random read at queue depth 1

fio --name=randread4k-qd1 \
  --filename=/root/fio-test.bin --size=1G --direct=1 \
  --rw=randread --bs=4k --iodepth=1 --numjobs=1 \
  --ioengine=libaio --runtime=60 --time_based --ramp_time=5 \
  --group_reporting --output-format=json > /root/randread4k-qd1.json

Queue depth 1 is latency, not throughput, and it is the number that governs interactive responsiveness: SSH login time, PHP request latency on an uncached page, a single-threaded MySQL query plan. Healthy NVMe delivers 12,000–18,000 IOPS at depth 1, which is 55–80 µs. Below 5,000 IOPS at depth 1 (200 µs or worse) you are looking at network-attached storage, and no amount of tuning inside the guest will change it.

Test 3: sustained random write for 5 minutes

fio --name=randwrite4k-5min \
  --filename=/root/fio-test.bin --size=4G --direct=1 \
  --rw=randwrite --bs=4k --iodepth=32 --numjobs=1 \
  --ioengine=libaio --runtime=300 --time_based \
  --group_reporting --output-format=json > /root/randwrite4k.json

This is the test that exposes burst credits and write throttles. Watch the IOPS in 10-second intervals rather than the aggregate; the useful signal is the shape of the curve, not its average. A device that starts at 30,000 IOPS and settles at 4,000 after 30 seconds has a token bucket, and the sustained figure is the one your database will live with.

fio --name=randwrite-timeline \
  --filename=/root/fio-test.bin --size=4G --direct=1 \
  --rw=randwrite --bs=4k --iodepth=32 --numjobs=1 \
  --ioengine=libaio --runtime=300 --time_based \
  --write_iops_log=/root/w-iops --log_avg_msec=10000 \
  --output-format=json
awk -F, '{printf "%6.0fs  %8d iops\n", $1/1000, $2}' /root/w-iops_iops.1.log

Test 4: sequential throughput at 1 MB blocks

fio --name=seqread1m \
  --filename=/root/fio-test.bin --size=4G --direct=1 \
  --rw=read --bs=1M --iodepth=16 --numjobs=1 \
  --ioengine=libaio --runtime=60 --time_based \
  --group_reporting --output-format=json

Sequential bandwidth matters for backups, log shipping and image pulls, and it is where virtualized and network storage most often underperform: 200–600 MB/s is normal for a single virtio queue on KVM NVMe, while network-backed volumes often cap near 120 MB/s per client. This test rarely fails on real NVMe and frequently fails on cloud block storage sold as “SSD”.

Reference results from a 4 vCPU KVM plan

The table below is what the four tests returned on an Ubuntu 24.04 guest with 8 GB RAM on KVM with host-side NVMe, run three times on separate nights to catch noisy-neighbour variance. Treat these as the healthy band, not as a leaderboard.

Test	Expected	Warning	Walk away
4k read, QD32 (IOPS)	40k–120k	15k–40k	< 10k
4k read, QD1 (IOPS)	12k–18k	5k–12k	< 4k
4k write sustained 5 min	> 8k	3k–8k	< 2k or 5× drop from burst
1M seq read (MB/s)	400–1500	150–400	< 120
Run-to-run variance	< 10%	10–30%	> 30%

Variance is the column most people ignore. An instance that hits 60,000 IOPS once and 22,000 the next night is sharing a saturated host, and the p99 latency in production will be defined by the bad nights. Run each test three times at the same hour and compute the spread before drawing a conclusion.

Removing guest-side noise before you blame the provider

Half of all “slow disk” reports are the guest fighting itself. Drop caches between runs, confirm no background job is running, and check whether the scheduler or a filesystem mount option is imposing overhead.

sync; echo 3 > /proc/sys/vm/drop_caches
# confirm nothing is using the device
iostat -x 5 3
# if journald or a package manager is holding I/O, stop it for the test
systemctl stop unattended-upgrades apt-daily.timer apt-daily-upgrade.timer

On ext4, testing an empty freshly-formatted filesystem skews results because allocation is fully sequential. Create the test file, write it once, delete it, and re-create it before measuring — or use a raw block device (--filename=/dev/vdb) to eliminate the filesystem entirely. Raw-device numbers are the cleanest comparison across providers, but never run a destructive test against a device with data; use a scratch volume.

Interpreting the numbers against your workload

• PostgreSQL / MySQL: the QD1 number drives read latency and the sustained write number drives checkpoint behaviour. A plan that fails the 5-minute write test will produce write stalls every checkpoint interval.
• Mail servers and log ingestion: sustained small writes, high concurrency. Fail test 3 and you have found your problem.
• Static web and CDN origin: mostly QD1 reads and page cache. Test 2 is the only one that matters.
• Backups and container image pulls: sequential bandwidth. Test 4 is the gate.

The feature that separates a usable plan from a marketing one is not peak IOPS — it is the sustained floor and the variance. If you are comparing providers on this basis, the storage and plan breakdown on the main site lists which tiers use local NVMe versus network volumes, which determines most of the numbers above before you run a single test. For the deeper kernel-side tuning that follows measurement, the technical guides index covers scheduler choice and I/O cgroup limits.

A 12-minute script to run the whole sequence

#!/usr/bin/env bash
set -euo pipefail
DEV=/root/fio-test.bin
run() { fio --name="$1" --filename="$DEV" --direct=1 --ioengine=libaio \
         --output-format=json --group_reporting "${@:2}" > "/root/$1.json"; }

fallocate -l 4G "$DEV"
run read-qd32 --size=2G --rw=randread --bs=4k --iodepth=32 --runtime=60 --time_based --ramp_time=10
run read-qd1  --size=1G --rw=randread --bs=4k --iodepth=1  --runtime=60 --time_based
run write-5m  --size=4G --rw=randwrite --bs=4k --iodepth=32 --runtime=300 --time_based
run seq-read  --size=4G --rw=read --bs=1M --iodepth=16 --runtime=60 --time_based

for f in read-qd32 read-qd1 write-5m seq-read; do
  printf "%-10s iops=%s  bw=%s MiB/s  lat=%s us\n" "$f" \
    "$(jq -r '.jobs[0].read.iops // .jobs[0].write.iops | floor' /root/$f.json)" \
    "$(jq -r '(.jobs[0].read.bw // .jobs[0].write.bw) / 1024 | floor' /root/$f.json)" \
    "$(jq -r '(.jobs[0].read.clat_ns.mean // .jobs[0].write.clat_ns.mean) / 1000 | floor' /root/$f.json)"
done
rm -f "$DEV"
echo done

Run it on a fresh instance before you commit to a billing cycle, run it again at 9 a.m. local time the next day, and compare. The second run tells you more about the host than the first, because it includes the neighbours who share your physical device.