A single-vCPU VPS can be perfectly healthy by every CPU metric and still drop packets under load. The usual cause is that the receive path is processed by softirqs on whichever CPU the driver raised them on, and on a one-vCPU guest there is exactly one place for that work to happen. This is not the same problem as a full softirq backlog; it is about where the work executes, how many queues exist, and whether the one CPU can keep up. This article covers the measurements that prove the bottleneck is in the softirq path, and the knobs that move it.<\/p>\n\n\n
When a frame arrives, the device raises an interrupt. The hardirq handler runs with interrupts disabled and is supposed to do the minimum: acknowledge the device, mask further interrupts, and schedule the deferred part. The deferred part is a softirq \u2014 NAPI polling in the The same received packet can therefore be charged to On a one-vCPU guest, Before tuning, prove that the ceiling is on the receive path and not on the sender, the link, or your application. The clean test is to generate traffic from a separate host and compare the rate at which packets arrive with the rate at which your process consumes them. If RX packets rise while your application’s request rate plateaus, and A virtio NIC in a KVM guest can present one or more virtqueues. Each queue can be bound to a vCPU, and in multi-queue mode each virtqueue’s interrupt has its own affinity. The mapping is visible in two places: the queue directories under the device, and the interrupt affinity mask in On a one-vCPU instance every mask will read Receive Packet Steering (RPS) emulates multiple receive queues in software by hashing flows across CPUs and queueing the deferred processing on a per-CPU backlog. Receive Flow Steering (RFS) goes further and steers a flow’s processing to the CPU where the application thread that owns the socket is running, improving cache locality.<\/p>\n\n On a single-vCPU guest neither does anything useful \u2014 there is one CPU to steer to \u2014 but the configuration still matters when you later resize. On a 2- or 4-vCPU plan, RPS is often the difference between one saturated core and even distribution, particularly when the virtio device exposes only one receive queue.<\/p>\n\n\n The definitive sign of a single-queue ceiling is that throughput stops scaling while packets keep arriving. In a guest this appears as: one virtqueue active in A larger ring absorbs bursts, but it does not reduce steady-state CPU per packet: per-packet cost is dominated by descriptor handling, skb allocation and the stack walk, which is why a flood of small messages hits a hard ceiling that large frames never reach. Larger rings buy burst headroom; more vCPUs buy packets per second.<\/p>\n\n\n A convincing diagnosis measures five things over one window: the per-CPU From there, the fix depends on what the platform allows. More vCPUs plus queue interrupts spread across them is the real solution; RPS helps when the driver exposes a single queue; coalescing and larger rings buy headroom. When none of the kernel levers is available \u2014 no multi-queue, no coalescing control, one vCPU \u2014 the constraint is the plan, and the VPS plans with more vCPUs and dedicated network queues<\/a> are the practical next step. Some of the apparent network limit is really storage or scheduler contention on the same core, covered in the kernel tuning guide for governors, swappiness and I\/O schedulers<\/a>.<\/p>\n\n\n On a single-vCPU VPS the softirq path is the only place receive work can run, so it saturates first and saturates quietly. Measuring where packets are counted, where interrupts land, and how the drop counters move is enough to prove the bottleneck without guesswork \u2014 and to know whether it is fixable from the guest or only by changing the plan.<\/p>","protected":false},"excerpt":{"rendered":" A single-vCPU VPS can be perfectly healthy by every CPU metric and still drop packets under load. The usual cause is that the receive path is processed by softirqs on…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"iawp_total_views":0,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-1542","post","type-post","status-publish","format-standard","hentry","category-vps-guides-tutorials"],"yoast_head":"\nNET_RX<\/code> vector \u2014 which runs with interrupts enabled, either right after the hardirq returns or in the ksoftirqd<\/code> thread when load is high.<\/p>\n\n%irq<\/code>, %softirq<\/code>, or to a kernel thread. The measurement question is how much CPU the receive path consumes per unit of traffic, and whether that work competes with your application threads.<\/p>\n\n\nmpstat -P ALL 2 5\ngrep -E '^(cpu|intr)' \/proc\/stat\nps -eo pid,comm,pcpu | grep -E 'ksoftirqd|kworker'\ncat \/proc\/interrupts | head -20<\/code><\/pre>\n\n\n%softirq<\/code> climbing while %sys<\/code> stays low is the signature to look for. Because there is only one CPU, ksoftirqd\/0<\/code> competing directly with your application threads is expected under sustained traffic, and it shows up as reduced application throughput rather than as a network error.<\/p>\n\n\nConfirm the Bottleneck Exists<\/h2>\n\n\n
%softirq<\/code> saturates on the single vCPU, the receive path is the limit.<\/p>\n\n\n# on the VPS: receive-side counters, sampled\nawk 'NR>2 {print $1, "rx_pkts="$3, "rx_drop="$4, "rx_err="$5}' \/proc\/net\/dev\nss -lntu | head\nnstat -az | grep -E 'InErrors|InDrops|RcvbufErrors|ListenDrops|TCPRcvQDrop'<\/code><\/pre>\n\n\n\nWhere the Interrupts Land<\/h2>\n\n\n
\/proc\/interrupts<\/code>. Reading both is the fastest way to see whether all RX work is funnelled onto one CPU.<\/p>\n\n\nls \/sys\/class\/net\/eth0\/queues\/\nethtool -l eth0\nethtool -S eth0 | head -30\nfor irq in $(awk '\/virtio\/{print $1}' \/proc\/interrupts | tr -d :); do\n echo "$irq: $(cat \/proc\/irq\/$irq\/smp_affinity_list)"\ndone<\/code><\/pre>\n\n\n0<\/code>, and that is the point: there is nowhere else to move the work. Interrupt affinity tuning is therefore not available, and what remains is reducing the cost per packet or adding a CPU. On instances with two or more vCPUs, spreading queue interrupts across vCPUs is the first and most effective fix.<\/p>\n\n\nRPS and RFS: Spreading Work Without Hardware Queues<\/h2>\n\n\n
# RPS: bitmask of allowed CPUs, one value per rx queue\necho f > \/sys\/class\/net\/eth0\/queues\/rx-0\/rps_cpus # CPUs 0-3\n# RFS: global flow table size and per-queue flow limit\nsysctl -w net.core.rps_sock_flow_entries=32768\necho 4096 > \/sys\/class\/net\/eth0\/queues\/rx-0\/rps_flow_cnt\n\n# verify the packets actually got steered\ncat \/proc\/softirqs | grep NET_RX\nmpstat -P ALL 2 5<\/code><\/pre>\n\n\n\nWhen One Queue Is the Ceiling<\/h2>\n\n\n
ethtool -S<\/code>, RX drops rising, %softirq<\/code> pinned near the CPU’s capacity, and application throughput flat. A useful check is to compare packets-per-second at two different frame sizes; if small frames collapse the rate far more than large ones, you are limited by per-packet processing cost rather than by bandwidth.<\/p>\n\n\nethtool -g eth0\nethtool -G eth0 rx 1024 tx 1024\n# batching indicators\nethtool -S eth0 | grep -iE 'packets|drops|queue'\ncat \/proc\/softirqs | awk '\/NET_RX\/{print}'<\/code><\/pre>\n\n\nPutting the Evidence Together<\/h2>\n\n\n
NET_RX<\/code> delta from \/proc\/softirqs<\/code>, the %softirq<\/code> share from mpstat<\/code>, the drop counters from nstat<\/code>, the queue count from ethtool -l<\/code>, and the application’s achieved request rate. When the softirq delta grows linearly with arriving packets while the request rate plateaus, the receive path is the bottleneck and no other explanation fits.<\/p>\n\nQuick Diagnostic Sequence<\/h2>\n\n\n\n
mpstat -P ALL<\/code> \u2014 is %softirq<\/code> the dominant term on the single vCPU?<\/li>\/proc\/softirqs<\/code> \u2014 does NET_RX<\/code> grow on one CPU only?<\/li>nstat<\/code> and \/proc\/net\/dev<\/code> \u2014 are drops at the socket, queue, or device level?<\/li>ethtool -l<\/code> and \/proc\/interrupts<\/code> \u2014 how many queues, and where are their interrupts?<\/li>