A Repeatable VPS Benchmark Method: Isolating CPU Steal, Disk p99 and Network Jitter

Most VPS benchmarks fail because they compare a single run against a screenshot from a review site, on different hardware, at a different time of day. A useful benchmark instead isolates one variable at a time and produces a number you can re-measure later. This method separates three signals that are routinely conflated: CPU steal, disk tail latency, and network jitter.

Establish a Baseline Under Idle Load

Run every test twice: once on an idle system, once under representative load. The gap between the two reveals contention, not capability. Record the following before you generate any load:

nproc
cat /proc/loadavg
vmstat 1 5
uptime
cat /sys/fs/cgroup/cpu.stat 2>/dev/null

From vmstat, the st column is CPU steal time — the percentage of cycles the hypervisor took away from your vCPU. Any sustained st above 5% during an idle baseline means the physical host is oversubscribed and your measurements will be noisy.

CPU: Distinguish Steal from Throttling

Use sysbench to measure raw throughput, but interpret it against steal:

sysbench cpu --cpu-max-prime=20000 --threads=$(nproc) run
sysbench cpu --cpu-max-prime=20000 run

The single-threaded run tests per-core performance; the multi-threaded run tests how well the instance scales. If events-per-second scales sublinearly while st climbs, the bottleneck is host contention, not your workload. Cross-check with perf stat -e cpu-clock,task-clock on a fixed loop to catch frequency scaling that sysbench hides.

Disk: Measure Tail Latency, Not Averages

Average IOPS is the least useful disk number on a shared NVMe host. The metric that determines user experience is the p99 and p99.9 latency at a realistic queue depth. fio reports percentiles directly:

fio --name=randread --ioengine=libaio --direct=1 \
  --rw=randread --bs=4k --iodepth=16 --numjobs=4 \
  --size=1G --runtime=60 --time_based \
  --lat_percentiles=1 --percentile_list=50:95:99:99.9 \
  --group_reporting

A healthy shared-NVMe instance at 4K random read, iodepth 16, shows p99 latencies in the low hundreds of microseconds. When p99 is 5–10x the p50, you are seeing queueing caused by a noisy neighbor rather than device speed. That distinction matters: a faster plan will not fix contention, but a different provider will.

Metric	Good sign	Red flag
p99 (4K randread)	< 500 µs	> 2 ms
p99/p50 ratio	< 3x	> 5x
IOPS vs plan spec	within 15%	< 60% of spec

Network: Measure Jitter, Not Just Throughput

A single iperf3 run gives you peak bandwidth, which tells you almost nothing about application performance. Run several short transfers and record the spread, then probe latency stability:

# On the peer: iperf3 -s
for i in 1 2 3 4 5; do
  iperf3 -c PEER --time 5 -P 4 | grep -E "SUM.*sender"
done
ping -c 200 PEER | tail -3
mtr --report --report-cycles 100 PEER

Watch the standard deviation, not the peak. A link that delivers 900 Mbit/s at peak but varies by 40% run-to-run has a bufferbloat or virtual-switch contention problem. mtr‘s loss column distinguishes loss at your gateway from loss further upstream — only the former is your provider’s responsibility.

Interpreting the Numbers in Context

No benchmark result exists in isolation. A single-threaded sysbench score of 4,000 events per second means nothing until you know the reference class — a modern shared-vCPU tier typically lands between 3,000 and 6,000, while a dedicated-core tier reaches two to three times that. Similarly, disk numbers must be read at the queue depth your workload actually generates; a web application rarely drives iodepth 32, so a benchmark at that depth can flatter an instance that disappoints in production.

The most common mistake is measuring a burst and assuming it is steady state. Shared hosts often burst above their sustained allocation for a few seconds to make interactive tests look good. Run every test for at least 60 seconds with --time_based and compare the first ten seconds against the last ten. A meaningful drop between them is throttling, and it will show up under real sustained load even if a short test hides it.

Repeatability Is the Point

Store your commands and results in a file so the next run is comparable. A benchmark you cannot repeat is an anecdote. When results degrade weeks later, re-running the identical commands turns “the server feels slow” into “p99 disk latency rose from 300 µs to 2.1 ms.”

• Same command, same parameters, same time of day.
• Record steal, load and ambient disk activity alongside each result.
• Version your scripts in git so historical numbers stay comparable.

Interpretation matters more than tooling. If you want reference points for what different tiers actually deliver, the provider benchmarks on this site compare CPU, disk and network side by side. For the underlying methodology, start from a machine that gives you honest steal numbers — some hosts are far more transparent about contention than others.

Turning Numbers into Decisions

1. Baseline on an idle system; note steal and load.
2. Run the same tests under load; measure the gap.
3. Isolate CPU, disk and network independently.
4. Look at tail percentiles and variance, never averages alone.
5. Re-measure on a schedule and keep the history.

Done consistently, this turns benchmarking from a one-off curiosity into a diagnostic tool you can rely on when something actually breaks.