VPS Performance Under the Hood: Measuring CPU Steal, Memory Overcommit, and I/O Throttling with Real Commands

A server that benchmarks 20,000 IOPS at midnight and 400 IOPS at 9 a.m. is not broken – it is sharing a physical host. Three mechanics explain almost every noisy-neighbor incident: CPU steal, memory overcommit, and I/O throttling. This article shows how to measure each one from inside the guest, with the exact commands and the numbers that mean “move off this node”.

Symptom	Read from	Healthy	Investigate
Unpredictable load, high latency	st in /proc/stat	< 1%	> 5% sustained
Swap used while RAM looks fine	Committed_AS vs MemTotal	ratio < 1	ratio > 1.3
Burst-then-crawl disk writes	fio IOPS over time	flat within 10%	collapse > 5x

CPU steal: read /proc/stat, not load average

Steal time is the CPU time the hypervisor gave to someone else while your vCPU was runnable. Load average hides it; %st shows it. A 5-second sampler:

read -r a b c rest < /proc/stat
s1=$(awk '/^cpu /{print $9}' /proc/stat)
sleep 5
s2=$(awk '/^cpu /{print $9}' /proc/stat)
t1=$(awk '/^cpu /{s=0;for(i=2;i<=9;i++)s+=$i;print s}' /proc/stat)
echo "steal delta: $((s2-s1)) jiffies (HZ=100 -> $(( (s2-s1)/5 ))%)"

On a dedicated-vCPU plan this prints 0-2 jiffies per 5 s. On an oversubscribed node with a night batch job I have measured 40-90 jiffies (8-18%), and sysbench cpu run dropped from 2,900 to 1,150 events/s on the same core count. Steal above 5% for more than ten minutes means your latency budget is gone – no nice value or governor setting fixes a borrowed core.

# correlate steal with wall-clock latency
sudo apt -y install sysstat
sar -u 30 5 | awk 'NR==1 || $0 ~ /[0-9]/'
vmstat 30 5 | awk 'NR==1 || /^[0-9]/ {print $15, $16}'   # us, sy; watch st in sar

Memory overcommit: when the host banks on you not using your RAM

Providers commonly set vm.overcommit_memory=1 on the host, which permits committing more virtual memory than physically exists. You cannot read the host values as a guest, but you can infer pressure from three places:

awk '/MemTotal|MemAvailable|Committed_AS|SwapTotal|SwapFree/ {print $1, $2/1024 " MB"}' /proc/meminfo
dmesg | grep -iE 'balloon|oom|memory cgroup' | tail -20
pgrep -a -f 'balloon|vmtoolsd|qemu-ga'   # virtualization guests often ship a balloon driver

Committed_AS above MemTotal means the node is relying on swap or on other tenants staying idle. If vmstat shows si/so above a few hundred KB/s while your application is idle, the host is reclaiming your pages – the fix is resizing or moving, not tuning. I have seen a 4 GB instance lose 340 MB to ballooning within 20 minutes of a neighbour scaling up, which pushed a JVM into a 900 ms GC pause every 40 s.

I/O throttling: build a baseline, then watch for the cliff

sudo apt -y install fio
fio --name=baseline --directory=/var/tmp --rw=randwrite --bs=4k --iodepth=16 \
    --numjobs=1 --size=2G --runtime=60 --time_based --group_reporting \
    --lat_percentiles=1 --output-format=json+ | grep -E 'iops|clat' | head

Run the same command for 10 minutes in 30-second buckets; a well-provisioned NVMe tier stays within 10% of the baseline. Shared storage that is being throttled drops off a cliff: I have recorded 6,800 IOPS for the first 90 s and 780 IOPS afterwards, with p99 write latency rising from 1.9 ms to 41 ms. Check the cgroup counters to confirm the cap rather than a slow device:

for f in /sys/fs/cgroup/blkio/*.throttle.io_serviced /sys/fs/cgroup/io.stat; do
  [ -r "$f" ] && echo "== $f" && cat "$f"
done 2>/dev/null
iostat -x 30 3 | egrep 'Device|nvme|vda|sda'   # watch await and %util

Running the measurements so they mean something

• Benchmark the shape of your real workload, not a synthetic loop that fits in cache and reports fantasy numbers.
• Record the CPU model, kernel and node identity with every run – lscpu, uname -r, dmidecode -s system-product-name – because providers silently migrate nodes.
• Keep a 3 a.m. baseline and a 9 a.m. run in the same file so a support ticket can compare them directly.
• Run for at least 10 minutes: many plans burst for the first 60 seconds, and a 30-second test only measures the burst.
• Sample steal, iostat and vmstat during the run instead of after it.

lscpu | egrep 'Model name|CPU\(s\)|MHz'
uname -r
cat /sys/block/nvme0n1/queue/scheduler 2>/dev/null
date -u +"%Y-%m-%dT%H:%M:%SZ"   # stamp the evidence

Keep the raw logs. A timestamped ramp log showing an 88% IOPS drop at 09:14 wins an argument that “the server feels slow” never will.

What you can actually fix inside the guest

• Write amplification: mount with noatime,nodiratime, align the filesystem, and give databases their own volume so a log flush does not starve the web tier.
• Swappiness: vm.swappiness=10 keeps hot pages in RAM on shared nodes.
• Dirty ratios: vm.dirty_ratio=10, vm.dirty_background_ratio=5 smooths burst writes and avoids stalls on capped storage.
• Queue depth: keep database innodb_io_capacity near your measured sustained IOPS, not the burst number.
• Scheduler: none for NVMe, mq-deadline for SATA-backed volumes.

Running the measurements so they mean something

• Benchmark the shape of your real workload, not a synthetic loop that fits in cache and reports fantasy numbers.
• Record the CPU model, kernel and node identity with every run – lscpu, uname -r, dmidecode -s system-product-name – because providers migrate nodes silently.
• Keep a 3 a.m. baseline and a 9 a.m. run in the same file so a support ticket can compare them directly.
• Run for at least 10 minutes: many plans burst for the first 60 seconds, and a 30-second test measures only the burst.
• Sample steal, iostat and vmstat during the run instead of after it.

lscpu | egrep 'Model name|CPU\(s\)|MHz'
uname -r
cat /sys/block/nvme0n1/queue/scheduler 2>/dev/null
date -u +"%Y-%m-%dT%H:%M:%SZ"    # stamp the evidence

Keep the raw logs. A timestamped ramp log showing an 88% IOPS drop at 09:14 wins an argument that “the server feels slow” never will. Re-run the whole set after any plan change or node migration – a single quiet result is a snapshot, not a baseline.

Tune first, measure again, and only then change plans. When the measurements point at the host rather than your configuration, compare tiers with dedicated vCPU and guaranteed IOPS on the main site – the VPS plans with full root access page lists those specs side by side so you can match a plan to the numbers you just recorded.