Evaluating a VPS Provider: A Measured Test Plan with Acceptance Thresholds

Provider marketing pages quote cores, RAM and a bandwidth number, and none of those tell you what a workload will feel like. Short of a paid trial, the way to find out is a structured test plan you run in the first days of an hourly instance: fixed measurements, known sample sizes, and acceptance thresholds written down before you buy. This is the plan we use, with the commands and the numbers that decide whether a provider stays on the shortlist.

Decide the thresholds before you see the results

Pick three to five criteria that map to your workload and give each a pass mark. For a general web host, sensible ones are: sustained CPU steal under 1 percent at idle and under 3 percent under load, 4k random read IOPS at least 70 percent of the advertised tier, p99 of a 1 KB response under 40 ms from your main user region, and no packet loss across a 30 minute ping sample.

Writing the acceptance table first prevents the common trap of rationalising a disappointing provider because the control panel is nice. Weight the criteria: for a database box, disk latency and steal dominate; for a media site, throughput and network route matter more. Track the candidates on a real baseline from a known-good machine you already run.

CPU: steal, scheduling jitter and sustained clocks

Run vmstat 1 60 on an idle instance and read the st column; anything above 1 percent on an otherwise quiet guest indicates noisy neighbours. Then load all vCPUs with stress-ng --cpu $(nproc) --timeout 300s and sample again. Also run sysbench cpu --threads=$(nproc) run twice, twenty minutes apart, to see whether burst capacity holds or collapses once credits are spent.

Steady-state clock is the other half. Read lscpu | grep MHz and grep MHz /proc/cpuinfo while loaded; hosted CPUs that fall to a low base clock under sustained load explain why a 4-vCPU plan underperforms a laptop. Compare per-core throughput figures, not core counts, between candidates — a 2-vCPU plan with real clocks often beats a 4-vCPU burst plan.

Disk: IOPS, latency distribution and steal from shared arrays

Use fio with a fixed job file so results compare across providers: fio --name=r4k --rw=randread --bs=4k --iodepth=32 --numjobs=4 --size=2G --runtime=60 --direct=1 --group_reporting. Repeat for random write and for 128k sequential. Report IOPS and, more importantly, the clat percentiles; a drive with good average latency but a p99 of 40 ms will produce application timeouts that never show in the mean.

Test for credit exhaustion by running the same job for ten minutes. Many tiers allow burst IOPS for a short window then drop to a steady fraction. If minute eight is half of minute one, you have measured the real plan. Compare that floor, not the burst figure, against your acceptance threshold.

Memory: honest allocation and ballooning behaviour

Confirm you actually get the RAM you paid for and that it is not ballooned away under host pressure. free -m should show nearly the full figure as available; then allocate 90 percent with stress-ng --vm 1 --vm-bytes 90% --timeout 120s while watching grep -E 'MemAvailable|SwapTotal' /proc/meminfo. Any unexpected swap-in during the test means the host is squeezing the guest.

Also check ECC reporting and NUMA layout with numactl --hardware. A single-node layout is normal on small plans; a two-node layout on a 2-vCPU plan usually means you are on a large shared host and should expect cross-node memory latency. Note whether nested virtualisation is exposed via grep -c vmx /proc/cpuinfo if you need Docker or KVM inside the instance.

Network: route quality, loss and throughput per stream

Measure from the region your users live in, not from your laptop. Run mtr -rwzc 200 target to a few real destinations and look for loss at your provider’s first hop — that is their problem, not the transit carrier’s. Then test single-stream and multi-stream throughput separately with iperf3; a plan that hits its headline rate only with eight parallel streams is fine for downloads and poor for a single database connection.

For a web host, add a latency test over the real path: 200 sequential HTTPS requests with curl -w '%{time_connect} %{time_appconnect} %{time_starttransfer}\n'. Report the 95th percentile of each phase. Rising time_connect with stable time_starttransfer points to a saturated or lossy path rather than a slow application.

Sample sizes and how long to test

Thirty seconds of iostat tells you nothing about a provider. Use a minimum of five minutes for disk, ten for sustained CPU, thirty minutes for ping and mtr, and a full day of passive collection if you are on hourly billing. Take every measurement at least three times at different hours and use the median; hosting performance varies with the neighbour load curve.

Collect the passive side with sar -u -r -n DEV 60 1440 writing to the archive, or a tiny systemd timer that appends vmstat and /proc/pressure/* to a log. The quiet hours matter: a provider that looks fine at 10 a.m. and shows 6 percent steal at 2 a.m. backup time will fail you during your own traffic peak.

Score the results and decide

Put every candidate in one table with the measured value, the threshold and a pass or fail mark. Reject any provider that fails a hard criterion twice regardless of price. Then rank the survivors by the weighted score and check the boring parts: snapshot pricing, IPv6 support, out-of-band console, and whether the support ticket about the p99 spike was answered with data or with a template.

Run the same plan again after ninety days. Tiers change quietly, and a provider that passes in month one can fail in month four once the host fills up. Our VPS comparison table records the spec and disk class of each plan so you can map your measurements onto the right tier, and the provider breakdown lists which ones publish real steady-state IOPS figures rather than burst peaks.