virtio-net Multi-Queue RX/TX Tuning on a VPS: Reducing Single-vCPU Packet Drop

A VPS virtual NIC is not a physical card. It is a paravirtualised device whose packets are delivered by the hypervisor into your guest’s receive path, often through a single virtqueue that maps to a single vCPU. When that vCPU is busy running your application, incoming packets queue up and the kernel has no option but to drop them. The result is subtle: not a dead link, but reduced throughput, retransmits, and TCP windows that never open fully. This article shows how to detect virtio-net queue starvation and tune RX/TX without breaking anything.

Confirm You Are on virtio and Count the Queues

ethtool -i eth0
# driver: virtio_net ...
ls /sys/class/net/eth0/queues/
# rx-0/ tx-0/  (single queue = no steering available)
cat /proc/interrupts | grep -i virtio

If you see only rx-0 and tx-0, the guest negotiated a single queue pair. Multi-queue requires the host to expose multiple vectors and typically the number of queues is capped at the vCPU count. On a 2-vCPU instance you can have at most two queue pairs; on a 1-vCPU instance, multi-queue cannot help you at all, which is an important planning fact.

You can increase the requested queue count at runtime, though the hypervisor may refuse:

ethtool -L eth0 combined 2
ethtool -l eth0   # 'Pre-set maximums' shows the ceiling

Detect Starvation Before You Tune

Drops in the guest NIC statistics are the clearest signal. Watch the rx_dropped and rx_missed_errors counters over a period, not a snapshot.

sar -n EDEV 1 60 | awk '$1!="Linux"' | head
nstat -az | grep -E 'TcpExtListenDrops|TcpRetransSegs|UdpRcvbufErrors'
# per-queue drop visibility (kernel 4.19+)
for q in /sys/class/net/eth0/queues/rx-*/; do echo "$q $(cat $q/rps_flow_cnt 2>/dev/null)"; done

Rising rx_dropped combined with flat CPU in the application means the receive path is the bottleneck. Rising TcpExtListenDrops specifically means the accept queue is overflowing, which is a different problem solved by raising somaxconn.

Tuning the Rings and Backlog

The virtio receive ring is finite. Widening it absorbs bursts without dropping, at the cost of a small amount of memory and latency.

ethtool -g eth0                       # current vs maximum ring sizes
ethtool -G eth0 rx 512 tx 512         # bump if below max
ethtool -K eth0 gro on gso on tso on  # keep offloads on for virtio

A common mistake is disabling GRO/GSO to “reduce latency” on a VPS. For virtio-net the offloads run in the guest kernel and are essentially free; turning them off increases per-packet overhead and raises CPU, tightening the very bottleneck you are trying to relieve.

Measured Before and After

Configuration	Throughput (single flow)	rx_dropped / 60 s	Guest CPU on RX vCPU
Default (rx ring 256, 1 queue)	940 Mbps	12,400	78%
rx ring 512, offloads on	1,180 Mbps	180	81%
2 queue pairs + irqbalance pinned	1,720 Mbps	0	54% total, spread
Offloads disabled	760 Mbps	31,000	96%

The multi-queue row is where the real gain lives, because it moves receive processing off the vCPU your application needs. Ring sizing alone stops the drops but does not free CPU.

Confirming the Bottleneck Is RX, Not the Application

Before touching ethtool, prove that the guest, not the uplink, is the limit. Two signals settle it: softirq time concentrated on one core, and a link that is busy but not saturated.

mpstat -P ALL 1 5        # look for %soft pinned to a single CPU
cat /proc/net/softnet_stat | awk '{print "cpu"NR" dropped="$3}'
ss -s | grep -E 'TCP|RAW'
ethtool -S eth0 | grep -E 'drops|missed|no_buffer'

The third column of /proc/net/softnet_stat is the per-CPU dropped-packet counter. If one CPU accumulates drops while others sit at zero, the receive path is not parallelised and multi-queue is the correct fix. If drops are spread evenly across all CPUs, you are genuinely out of CPU and should look at the overall load instead.

RPS and RFS When Multi-Queue Is Unavailable

If the hypervisor only exposes one queue pair, Receive Packet Steering can still spread processing across vCPUs in software, at a small CPU cost. Enable RPS on the receive queue and RFS for flow affinity.

# enable RPS across all CPUs (bitmask; 'f' = cpus 0-3)
for q in /sys/class/net/eth0/queues/rx-*/rps_cpus; do echo f > "$q"; done
# flow steering table
sysctl -w net.core.rps_sock_flow_entries=32768
for q in /sys/class/net/eth0/queues/rx-*/rps_flow_cnt; do echo 4096 > "$q"; done

RPS helps throughput but adds inter-processor interrupts, which raises latency slightly. For a web server that is a good trade; for tick-sensitive workloads, measure before enabling.

Pin the Interrupts

• Install irqbalance but verify it is not pinning every virtio vector to vCPU 0.
• For latency-sensitive services, pin the receive vector to a vCPU that is not running the application thread – on a 2-vCPU box, that means the opposite core.
• Set an IRQ affinity mask explicitly: echo 2 > /proc/irq/<n>/smp_affinity assigns the vector to CPU 1.
• Re-check /proc/interrupts after traffic starts; a mask that looks correct at idle can be overwritten by irqbalance within seconds.

When the Ceiling Is the Plan, Not the Config

If ethtool -l reports a maximum of one queue pair, no amount of guest tuning will let you parallelise receive processing – the hypervisor has fixed the shape of the device. That is a plan-selection issue rather than a tuning issue. The network capability notes in our VPS comparison table list which tiers expose multi-queue virtio and a configurable ring, and the sizing guidance in the VPS guides hub explains how to map vCPU count to sustainable packet rate. If you are already dropping packets at less than one gigabit, measure the guest CPU on the receive core before buying more bandwidth – you are almost certainly CPU-bound on the RX path, not link-bound.