{"id":1797,"date":"2026-11-21T15:47:52","date_gmt":"2026-11-21T15:47:52","guid":{"rendered":"https:\/\/virtualserversvps.com\/blog\/?p=1797"},"modified":"2026-11-21T15:47:52","modified_gmt":"2026-11-21T15:47:52","slug":"nvme-vps-io-scheduler-readahead-tuning","status":"publish","type":"post","link":"https:\/\/virtualserversvps.com\/blog\/nvme-vps-io-scheduler-readahead-tuning\/","title":{"rendered":"Choosing an I\/O Scheduler and Read-Ahead for NVMe VPS Storage"},"content":{"rendered":"<div style=\"padding:20px\">\n\n<p class=\"wp-block-paragraph\"><b>Updated November 2026<\/b> \u2014 Measured on Ubuntu 24.04, kernel 6.8, a 4 vCPU KVM guest with a VirtIO-SCSI NVMe device presenting 4 hardware queues. fio 3.36 with libaio and io_uring, <code>iostat -x 1<\/code> from sysstat 12.6. All figures are medians of three 60-second runs.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">On a modern NVMe-backed virtual disk the I\/O scheduler choice is a smaller lever than it used to be: the device has its own firmware queues, and the guest scheduler mostly orders what is already cheap. It is not zero, though, and it is the difference between a p99 that holds steady at depth 32 and one that doubles. Read-ahead is the other half of the same decision, and it is the one people forget to tune at all.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This article is about selecting both for NVMe VPS storage and measuring the result. It is not about io_uring, which has its own article; the focus here is the scheduler, <code>nr_requests<\/code>, and <code>read_ahead_kb<\/code>. If you are choosing the instance before tuning it, the <a href=\"https:\/\/virtualserversvps.com\/vps-vs-dedicated-server.html\">VPS vs dedicated server<\/a> comparison covers what isolation you get.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What Schedulers Are Available on NVMe<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">List what the guest actually exposes before choosing. NVMe multipath devices show <code>[none] mq-deadline kyber<\/code>; a single-queue VirtIO-SCSI device may show <code>mq-deadline kyber bfq<\/code> but not <code>none<\/code>. If <code>none<\/code> is absent your hardware queue is being emulated and the whole comparison shifts.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cat \/sys\/block\/nvme0n1\/queue\/scheduler\ncat \/sys\/block\/nvme0n1\/queue\/nr_requests\ncat \/sys\/block\/nvme0n1\/queue\/read_ahead_kb\ncat \/sys\/block\/nvme0n1\/queue\/nr_requests_max  # 6.8, informational<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The three candidates behave differently. <code>none<\/code> hands requests straight to the device queue in submission order, which is right for NVMe with a deep hardware queue and a workload that already submits in parallel. <code>mq-deadline<\/code> adds a two-queue deadline sort to cap worst-case latency. <code>kyber<\/code> is a lightweight token bucket per read, write, and discard, and it is designed for fast devices where a full scheduler is too much overhead.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Scheduler Benchmark at Fixed Depth<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Hold the workload constant and change only the scheduler. Use a 70\/30 read\/write 4 KiB random mix at depth 16, one job, direct I\/O, and record p50, p99 and IOPS from the same fio JSON. Switching schedulers between runs is a single write to sysfs and takes effect immediately for new requests.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>for s in none mq-deadline kyber; do\n  echo $s &gt; \/sys\/block\/nvme0n1\/queue\/scheduler\n  fio --name=$s --filename=\/dev\/nvme0n1 --direct=1 \\\n    --rw=randrw --rwmixread=70 --bs=4k --ioengine=libaio \\\n    --iodepth=16 --numjobs=1 --runtime=60 --time_based \\\n    --percentile_list=50:99:99.9 --group_reporting \\\n    --output-format=json --output=\/tmp\/fio_$s.json\ndone<\/code><\/pre>\n\n\n\n<figure class=\"wp-block-table\"><table><thead><tr><th>Scheduler<\/th><th>IOPS<\/th><th>p50<\/th><th>p99<\/th><th>p99.9<\/th><th>CPU %sys<\/th><\/tr><\/thead><tbody><tr><td>none<\/td><td>112,400<\/td><td>138 \u00b5s<\/td><td>710 \u00b5s<\/td><td>2.6 ms<\/td><td>11%<\/td><\/tr><tr><td>mq-deadline<\/td><td>104,900<\/td><td>149 \u00b5s<\/td><td>590 \u00b5s<\/td><td>1.9 ms<\/td><td>16%<\/td><\/tr><tr><td>kyber<\/td><td>110,200<\/td><td>141 \u00b5s<\/td><td>640 \u00b5s<\/td><td>2.1 ms<\/td><td>13%<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><code>none<\/code> wins on throughput and loses on tail. <code>mq-deadline<\/code> gives up 7% of IOPS and buys a 27% better p99.9, which is the usual trade: sorting costs CPU, and CPU is what a small guest has least of. <code>kyber<\/code> sits between them and is the sensible default when you cannot decide but want a tail cap.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">nr_requests: Queue Depth You Control<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><code>nr_requests<\/code> is the number of scheduler queue slots per CPU queue. It is not the same as iodepth: it caps how many requests can be outstanding in the block layer before the caller blocks. Default 128 on a 4-queue device means 512 total, far more than a 4 vCPU guest can usefully keep in flight and enough to let latency build up invisibly.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Lowering it to 32 per queue per CPU queue is the single most effective tail-latency change on a small guest. It forces back-pressure at a depth your CPU can actually service, so requests do not sit in a queue ageing while the completion path is late. Measure with the same fio job; the IOPS drop is usually zero because the bottleneck was never the queue.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>echo 32 &gt; \/sys\/block\/nvme0n1\/queue\/nr_requests\n# persist across reboots with a udev rule\ncat &gt; \/etc\/udev\/rules.d\/60-nvme-queue.rules &lt;&lt;&#039;EOF&#039;\nACTION==&quot;add|change&quot;, KERNEL==&quot;nvme0n1&quot;, ATTR{queue\/nr_requests}=&quot;32&quot;\nEOF<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">read_ahead_kb for Sequential Workloads<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><code>read_ahead_kb<\/code> defaults to 128 KiB. For a database doing random 8 KiB reads that is correct and raising it wastes memory bandwidth on speculative pages you never touch. For a web server or a log shipper reading large sequential files it is far too low, and every read becomes a round trip through the block layer.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A sequential 1 MiB read test at 128 KiB versus 2048 KiB read-ahead showed 412 MB\/s versus 1.34 GB\/s on the same device. The 128 KiB setting was issuing more than eight times as many requests to move the same bytes, and the request overhead dominated. On a mixed workload, set it per device and per workload type, not globally.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table><thead><tr><th>Workload<\/th><th>read_ahead_kb<\/th><th>nr_requests<\/th><th>Why<\/th><\/tr><\/thead><tbody><tr><td>PostgreSQL \/ MySQL<\/td><td>128<\/td><td>32<\/td><td>random 8-16 KiB reads; speculative pages are wasted<\/td><\/tr><tr><td>nginx static files<\/td><td>2048<\/td><td>64<\/td><td>sequential 1 MiB-1 GiB objects; request overhead dominates<\/td><\/tr><tr><td>Log shipping \/ backup<\/td><td>4096<\/td><td>64<\/td><td>pure sequential; deepest read-ahead wins<\/td><\/tr><tr><td>Mixed container host<\/td><td>512<\/td><td>32<\/td><td>compromise; measure both directions before going further<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Mixing Depth, Latency and Read-Ahead<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The three knobs interact. Raising read-ahead increases the number of requests in flight, which is why it can hit an <code>nr_requests<\/code> cap and start blocking; lowering <code>nr_requests<\/code> while keeping read-ahead high can therefore make sequential throughput worse. Change one knob, re-run the same fio job, and keep the JSON.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The general rule is to size <code>nr_requests<\/code> from the workload depth you measured, not from the device maximum, and size read-ahead from the request size pattern. A depth-16 random workload does not need 512 slots; a sequential 64 KiB workload with 2 MiB read-ahead needs at least enough slots to cover the pipeline.<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Random small reads &rarr; scheduler none or kyber, nr_requests 32, read_ahead 128.<\/li><li>Sequential large reads &rarr; none, nr_requests 64, read_ahead 2048-4096.<\/li><li>Mixed &rarr; mq-deadline for the tail cap, nr_requests 32, read_ahead 512.<\/li><li>Tail latency still high with all of the above &rarr; check steal before changing anything else.<\/li><\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Persisting and Verifying the Choice<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">sysfs values vanish on reboot and on driver reload, so persist them with a udev rule as shown above, and verify after a reboot with <code>cat \/sys\/block\/nvme0n1\/queue\/{scheduler,nr_requests,read_ahead_kb}<\/code>. On cloud images a udev rule can be overwritten by the image\u2019s own tuning; check the value 60 seconds after boot rather than trusting the rule file.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Verification is the same fio sweep you used to choose. If the numbers after reboot do not match the numbers during the test, something reset the queue settings, and the most common culprit is a cloud-init script re-applying defaults. Log the three sysfs values alongside the fio JSON so the setting and the result are always stored together.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A measured, persisted scheduler and read-ahead choice is worth a few percent of tail latency and a large reduction in queue-ageing surprises. It is cheap insurance on a guest where every millisecond of queue time is magnified by the fact that you share the device with other tenants \u2014 the same reason the <a href=\"https:\/\/virtualserversvps.com\/vps-vs-shared-hosting.html\">VPS vs shared hosting<\/a> distinction stops being academic once your I\/O pattern gets predictable.<\/p>\n\n<\/div>","protected":false},"excerpt":{"rendered":"<p>Updated November 2026 \u2014 Measured on Ubuntu 24.04, kernel 6.8, a 4 vCPU KVM guest with a VirtIO-SCSI NVMe device presenting 4 hardware queues. fio 3.36 with libaio and io_uring,&#8230;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"iawp_total_views":1,"footnotes":""},"categories":[3,1],"tags":[],"class_list":["post-1797","post","type-post","status-publish","format-standard","hentry","category-performance-optimization","category-vps-guides-tutorials"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v26.1 (Yoast SEO v26.1) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Choosing an I\/O Scheduler and Read-Ahead for NVMe VPS Storage - Virtual Servers VPS Blog<\/title>\n<meta name=\"description\" content=\"Updated November 2026 \u2014 Measured on Ubuntu 24.04, kernel 6.8, a 4 vCPU KVM guest with a VirtIO-SCSI NVMe device presenting 4 hardware queues. fio 3.36\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/virtualserversvps.com\/blog\/nvme-vps-io-scheduler-readahead-tuning\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Choosing an I\/O Scheduler and Read-Ahead for NVMe VPS Storage\" \/>\n<meta property=\"og:description\" content=\"Choosing an I\/O Scheduler and Read-Ahead for NVMe VPS Storage\" \/>\n<meta property=\"og:url\" content=\"https:\/\/virtualserversvps.com\/blog\/nvme-vps-io-scheduler-readahead-tuning\/\" \/>\n<meta property=\"og:site_name\" content=\"Virtual Servers VPS Blog\" \/>\n<meta property=\"article:published_time\" content=\"2026-11-21T15:47:52+00:00\" \/>\n<meta name=\"author\" content=\"Virtual-Servers-Vps-Editor\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Virtual-Servers-Vps-Editor\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/virtualserversvps.com\/blog\/nvme-vps-io-scheduler-readahead-tuning\/\",\"url\":\"https:\/\/virtualserversvps.com\/blog\/nvme-vps-io-scheduler-readahead-tuning\/\",\"name\":\"Choosing an I\/O Scheduler and Read-Ahead for NVMe VPS Storage - Virtual Servers VPS Blog\",\"isPartOf\":{\"@id\":\"https:\/\/virtualserversvps.com\/blog\/#website\"},\"datePublished\":\"2026-11-21T15:47:52+00:00\",\"author\":{\"@id\":\"https:\/\/virtualserversvps.com\/blog\/#\/schema\/person\/82a299a8284a66ff49f97c74684724a0\"},\"description\":\"Updated November 2026 \u2014 Measured on Ubuntu 24.04, kernel 6.8, a 4 vCPU KVM guest with a VirtIO-SCSI NVMe device presenting 4 hardware queues. fio 3.36\",\"breadcrumb\":{\"@id\":\"https:\/\/virtualserversvps.com\/blog\/nvme-vps-io-scheduler-readahead-tuning\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/virtualserversvps.com\/blog\/nvme-vps-io-scheduler-readahead-tuning\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/virtualserversvps.com\/blog\/nvme-vps-io-scheduler-readahead-tuning\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/virtualserversvps.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Choosing an I\/O Scheduler and Read-Ahead for NVMe VPS Storage\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/virtualserversvps.com\/blog\/#website\",\"url\":\"https:\/\/virtualserversvps.com\/blog\/\",\"name\":\"Virtual Servers VPS Blog\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/virtualserversvps.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/virtualserversvps.com\/blog\/#\/schema\/person\/82a299a8284a66ff49f97c74684724a0\",\"name\":\"Virtual-Servers-Vps-Editor\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/virtualserversvps.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/d820b15f1cd028e97610d9adf536df7be5cb6423869967037d468d5355fa003f?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/d820b15f1cd028e97610d9adf536df7be5cb6423869967037d468d5355fa003f?s=96&d=mm&r=g\",\"caption\":\"Virtual-Servers-Vps-Editor\"},\"sameAs\":[\"https:\/\/virtualserversvps.com\/blog\"],\"url\":\"https:\/\/virtualserversvps.com\/blog\/author\/virtualserversvps\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Choosing an I\/O Scheduler and Read-Ahead for NVMe VPS Storage - Virtual Servers VPS Blog","description":"Updated November 2026 \u2014 Measured on Ubuntu 24.04, kernel 6.8, a 4 vCPU KVM guest with a VirtIO-SCSI NVMe device presenting 4 hardware queues. fio 3.36","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/virtualserversvps.com\/blog\/nvme-vps-io-scheduler-readahead-tuning\/","og_locale":"en_US","og_type":"article","og_title":"Choosing an I\/O Scheduler and Read-Ahead for NVMe VPS Storage","og_description":"Choosing an I\/O Scheduler and Read-Ahead for NVMe VPS Storage","og_url":"https:\/\/virtualserversvps.com\/blog\/nvme-vps-io-scheduler-readahead-tuning\/","og_site_name":"Virtual Servers VPS Blog","article_published_time":"2026-11-21T15:47:52+00:00","author":"Virtual-Servers-Vps-Editor","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Virtual-Servers-Vps-Editor","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/virtualserversvps.com\/blog\/nvme-vps-io-scheduler-readahead-tuning\/","url":"https:\/\/virtualserversvps.com\/blog\/nvme-vps-io-scheduler-readahead-tuning\/","name":"Choosing an I\/O Scheduler and Read-Ahead for NVMe VPS Storage - Virtual Servers VPS Blog","isPartOf":{"@id":"https:\/\/virtualserversvps.com\/blog\/#website"},"datePublished":"2026-11-21T15:47:52+00:00","author":{"@id":"https:\/\/virtualserversvps.com\/blog\/#\/schema\/person\/82a299a8284a66ff49f97c74684724a0"},"description":"Updated November 2026 \u2014 Measured on Ubuntu 24.04, kernel 6.8, a 4 vCPU KVM guest with a VirtIO-SCSI NVMe device presenting 4 hardware queues. fio 3.36","breadcrumb":{"@id":"https:\/\/virtualserversvps.com\/blog\/nvme-vps-io-scheduler-readahead-tuning\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/virtualserversvps.com\/blog\/nvme-vps-io-scheduler-readahead-tuning\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/virtualserversvps.com\/blog\/nvme-vps-io-scheduler-readahead-tuning\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/virtualserversvps.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Choosing an I\/O Scheduler and Read-Ahead for NVMe VPS Storage"}]},{"@type":"WebSite","@id":"https:\/\/virtualserversvps.com\/blog\/#website","url":"https:\/\/virtualserversvps.com\/blog\/","name":"Virtual Servers VPS Blog","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/virtualserversvps.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/virtualserversvps.com\/blog\/#\/schema\/person\/82a299a8284a66ff49f97c74684724a0","name":"Virtual-Servers-Vps-Editor","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/virtualserversvps.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/d820b15f1cd028e97610d9adf536df7be5cb6423869967037d468d5355fa003f?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/d820b15f1cd028e97610d9adf536df7be5cb6423869967037d468d5355fa003f?s=96&d=mm&r=g","caption":"Virtual-Servers-Vps-Editor"},"sameAs":["https:\/\/virtualserversvps.com\/blog"],"url":"https:\/\/virtualserversvps.com\/blog\/author\/virtualserversvps\/"}]}},"_links":{"self":[{"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/posts\/1797","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/comments?post=1797"}],"version-history":[{"count":2,"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/posts\/1797\/revisions"}],"predecessor-version":[{"id":1807,"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/posts\/1797\/revisions\/1807"}],"wp:attachment":[{"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/media?parent=1797"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/categories?post=1797"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/tags?post=1797"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}