{"id":1216,"date":"2026-09-24T22:33:58","date_gmt":"2026-09-24T22:33:58","guid":{"rendered":"https:\/\/virtualserversvps.com\/blog\/?p=1216"},"modified":"2026-09-24T22:33:58","modified_gmt":"2026-09-24T22:33:58","slug":"profiling-cpu-hotspots-perf-flamegraphs-vps","status":"publish","type":"post","link":"https:\/\/virtualserversvps.com\/blog\/profiling-cpu-hotspots-perf-flamegraphs-vps\/","title":{"rendered":"Finding CPU Hotspots on a Small VPS with perf and Flamegraphs"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\"><code>top<\/code> shows a process at 100% CPU and tells you nothing about which function is burning it. That is the gap <code>perf<\/code> and flamegraphs close: instead of sampling at the process level, they sample at the instruction level and aggregate the samples into a single visual tree where the widest frames are the hottest code paths. This walkthrough builds that workflow on a small VPS, from installing <code>perf<\/code> to reading a flamegraph for the first time.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Installing perf and verifying it works<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><code>perf<\/code> ships with the kernel tools package. On Debian and Ubuntu the package is <code>linux-tools<\/code> and it must match your running kernel version, which is the first place people get stuck.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># Debian \/ Ubuntu\napt update\napt install -y linux-tools-common linux-tools-$(uname -r) linux-cloud-tools-$(uname -r)\n\n# RHEL \/ Alma \/ Rocky\ndnf install -y perf\n\n# sanity check\nperf --version\nperf stat -e cycles true<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">If <code>perf stat<\/code> reports <code>Permission denied<\/code> or an empty result, check that <code>kernel.perf_event_paranoid<\/code> is not too restrictive. Values above 2 block non-root sampling; setting it to 1 allows user-space profiling, which is what we need.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sysctl kernel.perf_event_paranoid\n# allow user-space sampling (not persistent; add to \/etc\/sysctl.d\/ to keep)\nsysctl -w kernel.perf_event_paranoid=1<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Sampling the right target on a small VPS<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">You do not profile the whole system. Profile the process that is busy, and choose the event that matches the symptom. Frequency of samples is a tradeoff: more samples give finer resolution but add overhead you must not ignore on a 1-2 vCPU instance.<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table><thead><tr><th>Goal<\/th><th>Command<\/th><th>Note<\/th><\/tr><\/thead><tbody><tr><td>Find CPU hot functions<\/td><td><code>perf record -F 99 -p PID -g -- sleep 30<\/code><\/td><td>99 Hz is the standard; 30s window<\/td><\/tr><tr><td>Whole-system profile<\/td><td><code>perf record -F 99 -a -g -- sleep 30<\/code><\/td><td>Higher overhead; use short windows<\/td><\/tr><tr><td>Count events only<\/td><td><code>perf stat -p PID sleep 10<\/code><\/td><td>Cheap; good for IPC and cache misses<\/td><\/tr><tr><td>Call graph depth<\/td><td>add <code>--call-graph dwarf<\/code><\/td><td>Better stacks, larger files, more overhead<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">On a two-vCPU server, an <code>-F 99<\/code> profile for thirty seconds costs well under one percent of a core. Avoid profiling for minutes at a time on a small instance; the perf ring buffer itself starts to compete for memory and cache. Short, targeted windows beat long, unfocused ones.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Generating the flamegraph<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">FlameGraph scripts turn the <code>perf.data<\/code> file into an SVG. Clone the tooling once, then the pipeline is three commands.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>git clone https:\/\/github.com\/brendangregg\/FlameGraph.git\n\n# 1. record\nperf record -F 99 -p $(pgrep -n php-fpm) -g -- sleep 30\n\n# 2. dump the stack samples\nperf script &gt; out.perf\n\n# 3. fold and render\nFlameGraph\/stackcollapse-perf.pl out.perf &gt; out.folded\nFlameGraph\/flamegraph.pl out.folded &gt; flamegraph.svg<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Copy the SVG to your laptop and open it in a browser. On a VPS with no desktop you can serve it over an existing nginx vhost or pull it down with <code>scp<\/code>. The file is self-contained HTML-wrapped SVG, so a single <code>scp<\/code> is all that is required.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Reading a flamegraph correctly<\/h2>\n\n\n\n<ul class=\"wp-block-list\"><li><strong>Width, not height, is cost.<\/strong> A wide frame consumes proportionally more CPU samples. Height is stack depth, not severity.<\/li><li><strong>Look for wide plateaus.<\/strong> A single function occupying 40% of total width is your target; ignore thin slivers.<\/li><li><strong>Follow the widest path first.<\/strong> Trace from the top down through the widest child at each level.<\/li><li><strong>Mind the colors.<\/strong> <code>flamegraph.pl<\/code> colors are random by default; do not read meaning into hue unless you pass a palette.<\/li><li><strong>Check for missing frames.<\/strong> If the graph is dominated by <code>[unknown]<\/code>, you are missing debug symbols or the right <code>--call-graph<\/code> mode.<\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">The most common real-world finding on a small web VPS is a wide frame under <code>regex<\/code> or <code>strstr<\/code> called from a templating or routing layer \u2014 a symptom of unindexed string work being repeated per request. The second most common is time in the kernel under <code>fib_lookup<\/code> or socket handling, which points at a network configuration problem rather than your code.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">A worked example: trimming a hot path<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Suppose the flamegraph shows 35% of samples under <code>preg_match<\/code> called from your framework&#8217;s router. That is a routing table being evaluated with regex on every request. The fix is usually to add an exact-match fast path before the regex list, or to compile the routes once at startup. After the change, re-record and diff the two graphs:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>perf record -F 99 -p $(pgrep -n php-fpm) -g -- sleep 30\nperf script | FlameGraph\/stackcollapse-perf.pl | FlameGraph\/flamegraph.pl &gt; after.svg\nperf diff old.perf after.perf   # function-level delta, if you kept the raw files<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Keep the raw <code>perf.data<\/code> files alongside the SVGs. <code>perf diff<\/code> gives you a numeric before\/after comparison that is far more convincing in a change review than two pictures.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Reducing perf&#8217;s own overhead<\/h2>\n\n\n\n<ul class=\"wp-block-list\"><li>Lower the sampling rate to <code>-F 49<\/code> when profiling a busy production process; the loss of resolution is usually irrelevant.<\/li><li>Prefer <code>--call-graph fp<\/code> over <code>dwarf<\/code> when binaries are compiled with frame pointers \u2014 stack unwinding becomes far cheaper.<\/li><li>Profile a single thread with <code>-t TID<\/code> rather than the whole process when only one worker is hot.<\/li><li>Write <code>perf.data<\/code> to a tmpfs mount to avoid competing with your workload for disk I\/O.<\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">If the workload is so tight that even 1% overhead is unacceptable, capture a short profile at the start of a maintenance window instead of continuously. Perf is a diagnostic, not an always-on monitor; use it to find the hotspot, then use lightweight counters to track whether your fix held.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">When to reach for perf<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Use <code>perf<\/code> when CPU is the bottleneck, when the process is busy but the profile is unknown, and when you have already ruled out I\/O wait and steal. It is a poor tool for diagnosing slow disk, network latency, or memory growth \u2014 those need <code>iostat<\/code>, <code>tcpdump<\/code>, or heap profiling respectively. Matching the tool to the bottleneck is half the work; if you are sizing new capacity to match a profiling result, review our <a href=\"https:\/\/virtualserversvps.com\/\">VPS plans<\/a> to buy only the cores the profile justifies, and compare <a href=\"https:\/\/virtualserversvps.com\/\">managed versus unmanaged options<\/a> if you want the platform to carry part of the tuning burden.<\/p>\n\n","protected":false},"excerpt":{"rendered":"<p>top shows a process at 100% CPU and tells you nothing about which function is burning it. That is the gap perf and flamegraphs close: instead of sampling at the&#8230;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"iawp_total_views":1,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-1216","post","type-post","status-publish","format-standard","hentry","category-vps-guides-tutorials"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v26.1 (Yoast SEO v26.1) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Finding CPU Hotspots on a Small VPS with perf and Flamegraphs - Virtual Servers VPS Blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/virtualserversvps.com\/blog\/profiling-cpu-hotspots-perf-flamegraphs-vps\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Finding CPU Hotspots on a Small VPS with perf and Flamegraphs\" \/>\n<meta property=\"og:description\" content=\"Finding CPU Hotspots on a Small VPS with perf and Flamegraphs\" \/>\n<meta property=\"og:url\" content=\"https:\/\/virtualserversvps.com\/blog\/profiling-cpu-hotspots-perf-flamegraphs-vps\/\" \/>\n<meta property=\"og:site_name\" content=\"Virtual Servers VPS Blog\" \/>\n<meta property=\"article:published_time\" content=\"2026-09-24T22:33:58+00:00\" \/>\n<meta name=\"author\" content=\"Virtual-Servers-Vps-Editor\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Virtual-Servers-Vps-Editor\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/virtualserversvps.com\/blog\/profiling-cpu-hotspots-perf-flamegraphs-vps\/\",\"url\":\"https:\/\/virtualserversvps.com\/blog\/profiling-cpu-hotspots-perf-flamegraphs-vps\/\",\"name\":\"Finding CPU Hotspots on a Small VPS with perf and Flamegraphs - Virtual Servers VPS Blog\",\"isPartOf\":{\"@id\":\"https:\/\/virtualserversvps.com\/blog\/#website\"},\"datePublished\":\"2026-09-24T22:33:58+00:00\",\"author\":{\"@id\":\"https:\/\/virtualserversvps.com\/blog\/#\/schema\/person\/82a299a8284a66ff49f97c74684724a0\"},\"breadcrumb\":{\"@id\":\"https:\/\/virtualserversvps.com\/blog\/profiling-cpu-hotspots-perf-flamegraphs-vps\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/virtualserversvps.com\/blog\/profiling-cpu-hotspots-perf-flamegraphs-vps\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/virtualserversvps.com\/blog\/profiling-cpu-hotspots-perf-flamegraphs-vps\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/virtualserversvps.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Finding CPU Hotspots on a Small VPS with perf and Flamegraphs\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/virtualserversvps.com\/blog\/#website\",\"url\":\"https:\/\/virtualserversvps.com\/blog\/\",\"name\":\"Virtual Servers VPS Blog\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/virtualserversvps.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/virtualserversvps.com\/blog\/#\/schema\/person\/82a299a8284a66ff49f97c74684724a0\",\"name\":\"Virtual-Servers-Vps-Editor\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/virtualserversvps.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/d820b15f1cd028e97610d9adf536df7be5cb6423869967037d468d5355fa003f?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/d820b15f1cd028e97610d9adf536df7be5cb6423869967037d468d5355fa003f?s=96&d=mm&r=g\",\"caption\":\"Virtual-Servers-Vps-Editor\"},\"sameAs\":[\"https:\/\/virtualserversvps.com\/blog\"],\"url\":\"https:\/\/virtualserversvps.com\/blog\/author\/virtualserversvps\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Finding CPU Hotspots on a Small VPS with perf and Flamegraphs - Virtual Servers VPS Blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/virtualserversvps.com\/blog\/profiling-cpu-hotspots-perf-flamegraphs-vps\/","og_locale":"en_US","og_type":"article","og_title":"Finding CPU Hotspots on a Small VPS with perf and Flamegraphs","og_description":"Finding CPU Hotspots on a Small VPS with perf and Flamegraphs","og_url":"https:\/\/virtualserversvps.com\/blog\/profiling-cpu-hotspots-perf-flamegraphs-vps\/","og_site_name":"Virtual Servers VPS Blog","article_published_time":"2026-09-24T22:33:58+00:00","author":"Virtual-Servers-Vps-Editor","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Virtual-Servers-Vps-Editor","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/virtualserversvps.com\/blog\/profiling-cpu-hotspots-perf-flamegraphs-vps\/","url":"https:\/\/virtualserversvps.com\/blog\/profiling-cpu-hotspots-perf-flamegraphs-vps\/","name":"Finding CPU Hotspots on a Small VPS with perf and Flamegraphs - Virtual Servers VPS Blog","isPartOf":{"@id":"https:\/\/virtualserversvps.com\/blog\/#website"},"datePublished":"2026-09-24T22:33:58+00:00","author":{"@id":"https:\/\/virtualserversvps.com\/blog\/#\/schema\/person\/82a299a8284a66ff49f97c74684724a0"},"breadcrumb":{"@id":"https:\/\/virtualserversvps.com\/blog\/profiling-cpu-hotspots-perf-flamegraphs-vps\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/virtualserversvps.com\/blog\/profiling-cpu-hotspots-perf-flamegraphs-vps\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/virtualserversvps.com\/blog\/profiling-cpu-hotspots-perf-flamegraphs-vps\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/virtualserversvps.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Finding CPU Hotspots on a Small VPS with perf and Flamegraphs"}]},{"@type":"WebSite","@id":"https:\/\/virtualserversvps.com\/blog\/#website","url":"https:\/\/virtualserversvps.com\/blog\/","name":"Virtual Servers VPS Blog","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/virtualserversvps.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/virtualserversvps.com\/blog\/#\/schema\/person\/82a299a8284a66ff49f97c74684724a0","name":"Virtual-Servers-Vps-Editor","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/virtualserversvps.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/d820b15f1cd028e97610d9adf536df7be5cb6423869967037d468d5355fa003f?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/d820b15f1cd028e97610d9adf536df7be5cb6423869967037d468d5355fa003f?s=96&d=mm&r=g","caption":"Virtual-Servers-Vps-Editor"},"sameAs":["https:\/\/virtualserversvps.com\/blog"],"url":"https:\/\/virtualserversvps.com\/blog\/author\/virtualserversvps\/"}]}},"_links":{"self":[{"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/posts\/1216","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/comments?post=1216"}],"version-history":[{"count":1,"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/posts\/1216\/revisions"}],"predecessor-version":[{"id":1219,"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/posts\/1216\/revisions\/1219"}],"wp:attachment":[{"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/media?parent=1216"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/categories?post=1216"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/virtualserversvps.com\/blog\/wp-json\/wp\/v2\/tags?post=1216"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}