When a Single VPS Stops Being Enough: Sizing Signals From CPU Steal, Load and Disk Wait

Every VPS owner eventually asks the same question: is this box too small, or is something on it broken? The two answers lead in opposite directions — one means spend money, the other means fix a query, a disk, or a memory limit. What separates them is not intuition but a small set of measurable numbers. This article defines the thresholds, shows the commands that produce them, and works through the arithmetic of when a single KVM guest stops being the right shape for a workload.

Start with steal time, not CPU usage

Inside a KVM guest, %st (steal) is time your vCPU was runnable but the host scheduler gave the physical core to someone else. It is the only metric that measures your provider rather than your application. A very small steal percentage is normal on a busy host; sustained steal means you are paying for cores you are not receiving.

Steal (15-min avg)InterpretationAction
< 1%Healthy, normal host contentionNone
1–3%Slightly noisy host, usually tolerableWatch trend over a week
3–8%Noticeable; latency variance appearsOpen a ticket with data
> 8% sustainedUnder-provisioned host or CPU oversoldMigrate or change plan
> 20% burstsNeighbour noise from a heavy tenantDocument and escalate
# instantaneous, per-vCPU
vmstat 1 10

# per-vCPU totals from boot
mpstat -P ALL 1 5

# 15-minute normalized average, greppable into a log
awk '/^cpu /{printf "steal=%.2f%%\n", $9}' /proc/stat

Steal is the first thing to check because it invalidates every other conclusion. If you are losing 15% of your CPU to a noisy neighbour, tuning your application is wasted effort. Collect vmstat 1 5 output over a few minutes at the same time of day for three days before drawing a conclusion — a single sample during a host migration proves nothing.

Load average versus vCPU count

Load average on Linux counts runnable and uninterruptible tasks. On a 2 vCPU guest, a load of 2.0 means every core has one task waiting — saturated but not necessarily pathological. A load of 6.0 means a queue is forming and latency is climbing. The useful measure is the ratio, not the absolute number:

Load / vCPUStateNotes
< 0.7Idle headroomVertical scaling likely unnecessary
0.7–1.0Efficiently usedThe target operating range
1.0–2.0Queuingp95 latency degrades; investigate cause
2.0–4.0OverloadedFix in-place if CPU-bound work exists
> 4.0SaturatedAdd vCPUs or reduce concurrency

Critically, load average cannot distinguish CPU-bound from I/O-blocked work. Both count. Use run-queue versus blocked from vmstat‘s r and b columns to tell them apart: a high r with low b is CPU pressure; a high b is disk or network wait, which adding vCPUs will not fix.

Disk wait: iowait lies, PSI does not

%iowait in top is widely misread. It counts time a CPU was idle while I/O was outstanding — on a virtualized guest this frequently includes time spent waiting on the hypervisor, so a low iowait does not mean the disk is fast. Pressure Stall Information, available on kernel 4.20 and later, is far more direct: it reports the percentage of wall-clock time tasks were stalled because of I/O, memory or CPU.

cat /proc/pressure/io
# some avg10=42.31 avg60=28.07 avg300=17.55 total=12839412

cat /proc/pressure/cpu
cat /proc/pressure/memory

# which process is doing the I/O
iostat -xz 1 5     # look at %util and await
pidstat -d 1 5     # per-process KB/s read and write

Interpret avg10 for I/O pressure as follows: below 5% the disk is not your constraint; 10–25% means noticeable stall on a portion of requests; above 40% sustained means the disk is actively throttling your application. Pair that with %util from iostat -x: sustained %util near 100% with await above 20 ms on SSD storage is a saturated device, whether or not the CPU looks busy.

Memory: the constraint most guests hit first

Page cache makes MemFree meaningless. What matters is MemAvailable and whether the kernel is reclaiming aggressively. Two signals mark the point where a guest no longer has enough RAM: sustained swap-out above roughly 200 KB/s, and a pgscan_direct counter that climbs — direct reclaim means allocations are blocking while the kernel frees pages, which shows up as latency spikes in every service, not just the memory-hungry one.

grep -E 'pgscan_direct|pgsteal_direct' /proc/vmstat
awk '/MemTotal|MemAvailable/{printf "%s %.0f MB\n",$1,$2/1024}' /proc/meminfo
smem -tk | sort -k4 -n | tail -12
cat /proc/pressure/memory    # full avg10 is the number that matters

If full avg10 for memory pressure is above 5% for more than a few minutes a day and the top consumers are your own services at sensible sizes, you are genuinely short of RAM. If the top consumer is a container with no limit or a PHP pool sized for a 16 GB machine, that is a configuration problem and more RAM just delays it.

Network and connection-level saturation

Two more limits are easy to miss. First, throughput: a 1 Gbps port shared among tenants rarely delivers 1 Gbps, and sar -n DEV 1 showing sustained use near the plan’s cap means you are limited by the port, not the CPU. Second, concurrency: if nginx has a fixed worker_connections of 512 per worker and you run one worker, everything above roughly 400 concurrent connections queues at the accept layer regardless of CPU available.

ss -s                     # socket summary: estab, timewait, orphaned
nstat -az | grep -iE 'drop|retrans|overflow'
sar -n DEV 1 5 | grep -v lo
sar -n TCP,ETCP 1 5       # retransmits indicate congestion or loss

Retransmits above 1% of segments, or a growing ListenOverflows in nstat, indicate the guest is dropping connection attempts — often a backlog setting rather than bandwidth. Raising net.core.somaxconn and nginx’s listen ... backlog=2048 fixes this for free, before any scaling decision.

Fix in place before you scale up

Before pricing a bigger instance, confirm none of these are true. Each has a measurable effect and takes minutes:

  • A database query without an index running thousands of times an hour — find it with pg_stat_statements or slow_query_log, then add the index.
  • PHP-FPM or app workers sized far above available memory, forcing the kernel to reclaim on every request.
  • Unbounded logging: a debug-level application log writing tens of GB a day will saturate a low-IOPS disk on its own.
  • No page cache in front of a dynamic site — a full-page cache typically removes 70–90% of PHP and database work.
  • Backups running at peak hours; move pg_dump and rsync jobs to a quiet window and watch I/O pressure drop.

Measure the same metrics before and after each change and record them. This log is what turns “feels faster” into evidence, and it makes the scaling decision later a matter of arithmetic rather than opinion.

Quantify the crossover point

A vertical scaling decision is justified when a resource is saturated *and* the workload is efficiently using what it has. Concretely: if load per vCPU is above 2.0 with steal under 2%, and after removing the fixable causes above the ratio stays there at your peak, the guest needs more cores. If I/O pressure is above 40% with await above 20 ms on a storage tier you cannot change, you need local NVMe or a different plan family. If memory pressure full avg10 is above 5% with no oversized consumers, you need RAM.

Then do the arithmetic on cost per unit of headroom. Doubling from 2 to 4 vCPU on a typical plan is a modest monthly increase and removes roughly half your peak queueing — usually the cheapest fix available, and far cheaper than a month of engineering time spent micro-optimising. Doubling again from 8 to 16 vCPU, for a workload that is not CPU-bound, buys nothing; that is the point where the shape of the deployment should change instead.

When vertical scaling runs out

There is always a ceiling. Past a certain size, per-core licensing costs, single-host failure domains and the difficulty of tuning one large machine outweigh the simplicity. The signals that you have reached it: peak load per vCPU above 2.0 at the largest plan you can afford, a maintenance window that no longer fits in your acceptable downtime, or one process that needs more than a third of total RAM. At that point the answer is horizontal — split the database onto its own guest, move static assets to object storage, or run two application nodes behind a load balancer.

Keep the metric history from the earlier steps. When you do split a role onto a second guest, the before-and-after numbers will tell you which split actually helped. Providers that expose per-vCPU steal and let you change plan size without reinstalling make this iterative process far less painful — see the managed versus unmanaged comparison for how much of this measurement work you would be handling yourself.

Leave a Reply