Diagnosing CPU Steal Time on a VPS: Finding Noisy-Neighbor Contention

Your VPS shows 40% CPU utilization, no swap pressure, and plenty of free RAM — yet response times spike at the same time every evening. When the capacity numbers say “plenty of headroom” but latency says otherwise, the culprit is often steal time: cycles your vCPU was ready to use but the hypervisor handed to another tenant on the same physical core. This is the noisy-neighbor problem, and it is diagnosable in a few minutes with tools already on your box.

What Steal Time Actually Measures

On a virtualized host, the guest kernel knows when its vCPU is runnable but not scheduled on a physical core. It cannot tell whether the hypervisor is idle, busy with another tenant, or handling host-level work — it only records the wait. That wait is exported as steal in /proc/stat, shown as %st in top, and as the st column in vmstat. On a well-behaved host it stays under 1-2%. Above 5% you are losing meaningful throughput to someone else.

Steal is not your process being slow. It is your process not being allowed to run. That distinction is why a look at your own application metrics alone will never reveal it — you have to read the virtualization counters.

Reading /proc/stat by Hand

The raw counter gives you an exact picture without relying on a monitoring tool’s aggregation or a single tool’s rounding.

# Fields for cpu0: user nice system idle iowait irq softirq steal guest gnice
cat /proc/stat | grep '^cpu'

# Compute steal as a percentage over a 10-second window
awk '/^cpu / {u=$2+$3; s=$4; i=$5; w=$6; ir=$7; si=$8; st=$9;
  t=u+s+i+w+ir+si+st; printf "steal=%.2f%%\n", st*100/t}' /proc/stat
sleep 10
awk '/^cpu / {u=$2+$3; s=$4; i=$5; w=$6; ir=$7; si=$8; st=$9;
  t=u+s+i+w+ir+si+st; printf "steal=%.2f%%\n", st*100/t}' /proc/stat

Run it during a busy period, not at 3 a.m. A single reading is noise; sample every 10 seconds for five minutes and look at the 95th percentile, because steal arrives in bursts.

Cross-Checking with vmstat, top, and mpstat

Three tools, three views of the same counter. Agreement across them rules out a single tool’s bug.

# Column 15 is st (steal %), column 17 is id (idle %)
vmstat 1 20 | awk 'NR==1 || NR>2 {printf "%s st=%s id=%s\n", $1, $15, $17}'

# Per-CPU steal — a single hot vCPU is a strong noisy-neighbor signal
mpstat -P ALL 2 5

# top's summary line: us sy ni id wa hi si st
top -bn1 | grep '%Cpu'

The mpstat -P ALL output is the most revealing. If one or two vCPUs consistently show stealing while others are idle, you are being descheduled on specific physical threads, which is the classic signature of an oversubscribed host placing another tenant on your core.

Distinguishing Steal from Your Own Throttling

Before blaming a neighbor, rule out self-inflicted limits. Providers enforce CPU credits and disk IOPS caps per instance; hitting those looks similar in a latency graph but has a different root cause.

Symptom	Steal (neighbor)	Your throttle
vmstat st%	High (5%+)	Near zero
iowait (wa)	Normal	Elevated
Pattern	Unpredictable bursts	Trails sustained load
AWS/credit-style burst	Unaffected	Drains after minutes
Disk latency	Normal	fio shows a hard wall at the same IOPS number

Test the throttle hypothesis with a bounded disk run. If fio plateaus at exactly the same IOPS every time regardless of load, that is a provider cap, not contention:

fio --name=cap --rw=randread --bs=4k --iodepth=64 --runtime=30 \
    --time_based --output-format=json | jq '.jobs[0].read.iops'

Correlating Steal with Application Latency

Steal matters only if it shows up downstream. Use sar to capture history and correlate the two, so your next support ticket has evidence rather than a hunch.

# Install sysstat, then collect every minute
sudo systemctl enable --now sysstat
sar -u 1 5            # %steal in the last field
sar -u -f /var/log/sysstat/sa$(date +%d) | awk '$0 ~ /^[0-9]/ {print $1, $8}'

# Compare against your app p99 (example: nginx upstream time)
awk '{print $0}' /var/log/nginx/upstream_p99.log | tail -60

If p99 latency and steal% rise together on the same timestamps, you have a solid case. If p99 spikes with steal flat, the bottleneck is in your stack, not the host — look at query plans, connection pool exhaustion, or GC pauses.

What to Do When You Confirm Contention

• Open a ticket with data. Paste the mpstat output and timestamps. Vague complaints get generic replies; per-CPU steal tables get migrations.
• Ask to be moved to a quieter host. Most providers will live-migrate an instance on documented repeated contention.
• Pin your latency-critical process. taskset -c 2,3 postgres keeps it on the least-contended vCPUs.
• Move the sensitive tier to dedicated cores. Database and broker workloads belong on instances with pinned vCPUs — the trade-offs are covered in our comparison of VPS and dedicated server isolation.
• Right-size the plan. More vCPUs on an oversubscribed host do not help; move to a host with a lower oversubscription ratio instead.

Steal in Containers vs KVM Guests

On a KVM guest the steal counter is exported by the hypervisor and is trustworthy. On container-based plans (LXC/OpenVZ) “steal” may read zero even while you are being throttled, because a cgroup CPU quota simply stops your processes without reporting a wait. On those platforms, read the throttling counters instead and treat nonzero values under load as the container equivalent of steal:

cat /sys/fs/cgroup/cpu.stat
# watch nr_throttled and throttled_usec climbing while your app is slow

Takeaway

Steal time is the difference between owning cycles and renting them. Measure it during peak hours, cross-check it per vCPU, confirm it maps to real latency, and then act — a ticket, a migration, or a move to an instance class that gives you the cores outright. If repeated contention keeps costing you, review the VPS plans built around lower oversubscription before you scale the instance you already have.