Verifying VPS Network Performance: iperf3, mtr and Disk Floor Checks

An iperf3 run that reports 9.4 Gbit/s between two VPS instances in the same region tells you almost nothing about how a user in Frankfurt experiences your API. Single-stream throughput between two well-provisioned endpoints measures the least interesting path: no congestion, short RTT, large send buffers. The measurements that predict user experience are single-stream throughput over a realistic RTT, packet loss from mtr, and the disk floor that limits what the network can deliver when data must be flushed to storage.

This article is a verification checklist for a new instance: confirm the advertised network rate, find the real single-stream ceiling, read loss and jitter from mtr, and establish the disk floor so that a network test is not secretly measuring storage.

Baseline: Confirm the Interface and Link Speed

Before generating traffic, verify what the guest actually has. A virtio interface negotiates speed with the host, and while the host often virtualises the number, a mismatch in the guest is a genuine misconfiguration. Check the driver, ring sizes and offload state.

ethtool eth0 | grep -E 'Speed|Duplex|Link detected'
ethtool -i eth0              # driver and firmware
ethtool -k eth0 | grep -E 'tso|gso|gro|checksum'
ethtool -S eth0 | grep -E 'tx_queue_stopped|rx_drops|tx_dropped'
ip -s link show eth0         # errors, dropped, overrun

Non-zero tx_queue_stopped or growing dropped on the interface counters during a test points at a queue limit rather than the network path. Raise net.core.txqueuelen or check qdisc state with tc -s qdisc show dev eth0. Also confirm net.core.rmem_max and net.ipv4.tcp_rmem are large enough for a long-fat path; default 208 KiB receive windows cap single-stream throughput at roughly 25 Mbit/s over 60 ms RTT.

The bandwidth-delay product is the arithmetic that governs it: 1 Gbit/s × 60 ms RTT needs about 7.5 MB in flight. Without tcp_wmem and tcp_rmem maxima near that value, no amount of link capacity helps.

Multi-Stream vs Single-Stream iperf3

Run iperf3 in both modes and compare. Multi-stream saturates the link; single-stream reflects what one file transfer or one HTTP/2 connection sees. The gap between them is where most “the network is slow” complaints live.

# receiver
iperf3 -s

# sender: 8 streams, 30 s
iperf3 -c 203.0.113.10 -t 30 -P 8

# sender: naive single stream, 30 s
iperf3 -c 203.0.113.10 -t 30

# sender: single stream with an 8 MB window (long-fat path)
iperf3 -c 203.0.113.10 -t 30 -w 8M --get-server-output

# single stream in reverse (download direction)
iperf3 -c 203.0.113.10 -t 30 -R -w 8M

# UDP test for loss and jitter at 1 Gbit/s
iperf3 -c 203.0.113.10 -u -b 1G -t 20 --get-server-output

A healthy result looks like: 8 streams reaching near the provisioned rate, a single stream with an 8 MB window reaching 70–90% of the multi-stream figure, and the naive single stream with default window lower still. The UDP run reports lost packets at the receiver; more than 0.1% loss at a rate well below the provisioned cap indicates a problem, while loss at the cap is expected policing.

Reading mtr for Loss and Route Changes

mtr combines traceroute and ping, accumulating per-hop statistics. The mistake is reading loss at an intermediate hop as a problem; routers deprioritise ICMP to their own control plane, so loss at hop 4 with zero loss at hop 12 is usually reporting behaviour, not packet loss.

mtr -rwzbc 100 203.0.113.10 | tee /root/mtr-out.txt
# -r report, -w wide, -z ASN lookup, -b IPs, -c 100 cycles

# compare both directions when you control both ends
mtr -rwc 100 --tcp --port 443 target.example.com

Interpret as follows: loss that begins at a hop and persists to the destination is real; loss that appears at one hop and disappears later is ICMP rate limiting. High StDev with low average on intermediate hops is jitter from control-plane processing. For a TCP service, run mtr --tcp --port 443 so the probes follow the same forwarding path as real traffic rather than the ICMP path, which can differ.

Route asymmetry is common on cloud networks: the forward path may be three hops and the return path nine. Test from both ends if you can, and compare mtr output from the user side against the server side. Where you cannot, a tracepath run from the instance to the user’s network with tracepath -n target gives the reverse view including MTU discovery results.

Establishing the Disk Floor

Many network tests are accidentally disk tests. A scp of large files, an HTTP upload to disk, or an iperf3 run writing to a file with --file can be limited by storage. Measure the floor first so you know which number is the ceiling.

# sequential write throughput
fio --name=seqw --filename=/mnt/scratch/net-floor --size=1G \
    --bs=1M --rw=write --direct=1 --ioengine=libaio --iodepth=8

# sequential read throughput
fio --name=seqr --filename=/mnt/scratch/net-floor --size=1G \
    --bs=1M --rw=read --direct=1 --ioengine=libaio --iodepth=8

# how long does an fsync of 64 MB take?
dd if=/dev/zero of=/mnt/scratch/flush-test bs=1M count=64 conv=fsync oflag=direct 2>&1 | tail -1

A 1 Gbit/s link needs 125 MB/s of sustained write to keep up; if the disk floor is 90 MB/s, any receive-to-disk workload caps at 720 Mbit/s regardless of network capacity. That is not a network fault, and filing it as one wastes time. Record the floor next to the network figures so future comparisons keep the two separate.

Separating Storage Backend from Local Disk

On a VPS, the storage path often traverses the network too—network block storage or a replicated volume. In that case the disk floor is itself network-sensitive, and network tests compete with disk traffic for the same NIC. Run them sequentially, never concurrently, and note the storage backend in your records.

A quick way to see whether disk I/O is traversing the network is to watch /proc/net/dev byte counters while running an fio test with no other traffic. If interface counters climb in step with disk throughput, the volume is remote.

A Repeatable Verification Sequence

Run the checks in a fixed order so results stay comparable across instances and across time. Order matters because later tests are sensitive to earlier load, and CPU contention from a previous benchmark skews the next one.

• Idle baseline: ip -s link, ethtool -S, cat /proc/net/dev snapshot.
• Disk floor: sequential write and read with fio --direct=1, then the 64 MB fsync.
• Single-stream iperf3 with an 8 MB window, both directions.
• Multi-stream iperf3 at the expected core count.
• UDP iperf3 at the provisioned rate for loss and jitter.
• mtr -rwzbc 100 to three destinations on different continents.
• Compare RSS/softirq CPU during each run via mpstat -P ALL 1 and sar -n DEV 1.

Keep the raw outputs. A timestamped directory of iperf3 JSON, mtr text and fio JSON is the only way to tell whether a change is real or a host-load artefact. Re-run quarterly; a degradation that appears across several checks is actionable, while a single bad run usually is not.

The distinction between “the network is capped” and “this instance is small” matters when deciding whether to resize. If packet loss is clean and single-stream throughput is limited by CPU spent in softirq, more cores will help and more bandwidth will not—compare managed vs unmanaged in our guide if you want the platform team to own that diagnosis. If you are still choosing a plan, our VPS vs dedicated server comparison covers when a shared NIC is the real constraint.