Diagnosing Packet Loss on a VPS with mtr and pathping: Per-Hop Truth vs Noise

Packet loss that shows up only as slow application performance is one of the hardest faults to localise, because the loss may occur on your instance, in the provider’s fabric, at a transit peer, or at the destination — and the tools built for the job are frequently misread. mtr and pathping both blend continuous probing with per-hop statistics, but their defaults hide the two facts you actually need: whether loss is real or a control-plane artifact, and whether it persists to the final hop. This article covers how to run both correctly against a VPS, how to distinguish per-hop false positives from genuine path loss, and how to correlate the result with interface counters.

Why a Single ping Run Misleads

A 10-packet ping gives you an average that hides burst loss entirely. A path can drop 20% of packets in short bursts during congestion while a short sample reports near-zero loss and a healthy RTT. Use a sustained probe with timestamps and a longer window before concluding anything:

# 300 packets, 0.2s interval, timestamped — reveals bursts a plain ping misses
ping -D -i 0.2 -c 300 destination.example.com | tee /tmp/ping.log

# Summarise loss and jitter from the log
tail -3 /tmp/ping.log

Reading mtr Without Falling for Per-Hop Loss

mtr probes every hop along the path repeatedly. Modern versions can run in report mode, send both ICMP and TCP, and separate forward from return paths. The critical interpretation rule: loss at an intermediate hop that does not continue to the final hop is almost always a control-plane artifact. Routers deprioritise ICMP generation for TTL-expired packets under load, so they under-report. If hop 7 shows 40% loss but hop 8 and the destination show 0%, there is no real forwarding loss.

# Report mode, TCP SYN probes (better through firewalls/anycast), 100 cycles
mtr --report --report-cycles 100 --tcp --port 443 destination.example.com

# JSON output for scripting and diffing before/after
mtr --report --report-cycles 200 --json destination.example.com > /tmp/mtr.json

# Show both directions if the source supports it
mtr --report --report-cycles 100 --bidir destination.example.com

Compare the loss column top-to-bottom. Real loss that originates at some hop will be present at that hop and every hop after it, including the destination. Isolated loss at a single middle hop, with clean hops on both sides, is noise. TCP-mode probes are useful here because many transit routers rate-limit ICMP harder than TCP, inflating apparent ICMP loss.

pathping: Windows-Style Per-Hop Stats on Linux/Windows

pathping combines tracert discovery with a sustained statistical phase, sending many probes per hop and reporting loss and latency per hop. On Windows it is built in; on Linux install mtr or the pathping equivalent from your distro. The statistical phase is what matters — the initial traceroute phase only discovers the path.

# Windows: 100 probes per hop, 1s interval, no name resolution for speed
pathping -q 100 -p 1000 -n destination.example.com

# Linux equivalent via mtr with full per-hop counts
mtr --report --report-cycles 100 --no-dns destination.example.com

The output separates “this node/link” loss (loss at that specific hop) from “source to here” cumulative loss. As with mtr, trust the cumulative column at the final hop. A hop showing high “this node” loss but negligible cumulative loss at the destination is a router generating ICMP slowly, not a broken link.

Correlating with Interface and Stack Counters

Path diagnostics tell you where; the guest kernel tells you whether the loss is inbound, outbound, or dropped locally. Read these counters before and after a test window:

# NIC-level drops, errors, and ring buffer overruns
ethtool -S eth0 | grep -E 'drop|error|miss|fifo|overrun|nobuf'

# TCP retransmits indicate loss on the actual flow, not just probes
nstat -az | grep -E 'Retrans|TcpExt.*Lost|TcpExt.*Drop'

# Per-socket retransmit detail
ss -tin dst destination.example.com

# Softirq backlog drops (packets dropped before the stack saw them)
awk '{print NR-1, strtonum("0x"$2)}' /proc/net/softnet_stat

If retrans counters climb but ethtool -S shows no receive drops, loss is on the path or at the peer, not on your instance. If softnet_stat shows backlog drops on one CPU, you have a local receive-side bottleneck (see our tuning of per-queue softirq processing) rather than a network fault. If rx_dropped grows on the NIC, ring buffer pressure is the cause and raising the rx ring is the fix.

# Enlarge the RX ring if rx_dropped is climbing
ethtool -g eth0        # current vs maximum ring size
ethtool -G eth0 rx 4096

Is It the Path or the Provider?

Run the same probe to three destinations: a well-known anycast resolver, your own provider’s looking-glass target, and the actual service endpoint. If loss appears only toward the real endpoint, suspect the far side. If loss appears toward every destination from your instance, suspect your instance or your provider’s first hop — check ethtool -S and softnet_stat first, then open a ticket with the mtr JSON attached. If loss appears on a single middle hop toward all destinations but not in the final-hop cumulative column, it is cosmetic and you should ignore it.

Finally, document a baseline. Capture mtr JSON from a healthy instance during normal operation so that when an incident begins you have a comparison rather than a single ambiguous snapshot. When choosing infrastructure to run these probes from — or when comparing providers whose network paths you will be measuring — the main VPS comparison resources cover how to pick instances by measured network behaviour rather than the marketing spec sheet.

Leave a Reply