How to Set Up a Virtual Machine on Your VPS: KVM Installation, Resource Allocation, and Performance Tuning

Running a virtual machine inside your VPS — known as nested virtualization — lets you create isolated environments with their own operating system kernels, network stacks, and resource limits. This is useful for testing software across multiple Linux distributions, running legacy applications that require specific kernel versions, or providing isolated development environments for your team. Before choosing a provider for nested virtualization workloads, compare VPS performance across providers to ensure your plan delivers enough CPU and memory headroom. This guide walks through installing KVM, allocating resources, tuning the host for performance, and managing your VMs efficiently.

Understanding Nested Virtualization

Nested virtualization means running a hypervisor (like KVM) inside a virtual machine that is itself hosted on a hypervisor. Most modern VPS providers that use KVM at the host level support nested virtualization, but some disable it for security or performance reasons. The key requirement for nested KVM is that your CPU exposes hardware virtualization extensions (vmx for Intel, svm for AMD) and that the host hypervisor passes those extensions through to your VPS instance. Providers like Vultr, Linode, Hetzner, and DigitalOcean support nested KVM on most of their plans. Providers using OpenVZ or LXC containers do not support nested KVM at all.

Step 1: Verify Nested Virtualization Support

Run these checks to confirm your VPS supports nested KVM:

# Check for virtualization flags
grep --color=auto 'vmx\|svm' /proc/cpuinfo

# Check if kvm modules can be loaded
sudo modprobe kvm_intel  # or kvm_amd
lsmod | grep kvm

# Check for /dev/kvm device
ls -la /dev/kvm

If /dev/kvm exists and the kvm_intel or kvm_amd module loads without errors, you are ready to proceed. If the module fails to load, your provider does not support nested virtualization — consider switching to a plan or provider that does.

Step 2: Install KVM and Management Tools

Install the complete virtualization stack on your VPS:

Ubuntu/Debian:

sudo apt update
sudo apt install qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils virtinst virt-manager -y
sudo systemctl enable --now libvirtd
sudo usermod -aG libvirt $USER

Rocky Linux/RHEL 9:

sudo dnf install @virtualization -y
sudo systemctl enable --now libvirtd
sudo usermod -aG libvirt $USER

Log out and back in for group membership changes to apply.

Step 3: Configure Resource Allocation for Nested VMs

When running nested VMs, resource planning is critical because everything is double-billed — the host VPS has finite RAM and CPU, and each nested VM consumes from that pool. A good rule of thumb is to leave at least 1 GB of RAM and 1 vCPU for the host operating system and libvirt daemon, then distribute the remaining resources across your VMs.

Use virsh to configure CPU and memory limits on your VMs:

# Set maximum memory for a VM (8 GB)
sudo virsh setmaxmem vm-name --size 8G --config

# Set current memory allocation
sudo virsh setmem vm-name --size 4G --config

# Pin vCPUs to specific host cores for consistent performance
sudo virsh vcpupin vm-name 0 0
sudo virsh vcpupin vm-name 1 1

Step 4: Create a Nested Virtual Machine

Use virt-install to create your first nested VM. The key parameter for nested virtualization is the CPU mode — set it to host-passthrough so the nested VM sees the same CPU features as the host:

sudo virt-install \
  --name ubuntu-nested \
  --ram 2048 \
  --vcpus 2 \
  --cpu host-passthrough \
  --disk size=15 \
  --cdrom /var/lib/libvirt/boot/ubuntu-24.04.iso \
  --os-variant ubuntu24.04 \
  --network network=default \
  --graphics vnc \
  --console pty,target_type=serial

The --cpu host-passthrough flag is essential — without it, the nested VM will not detect KVM support and will fall back to emulated (slow) CPU mode.

Step 5: Tune the Host for Nested Virtualization Performance

Nested virtualization adds overhead because each hypervisor call traverses two layers. These tuning steps minimize the performance penalty:

Use huge pages: KVM can map guest memory using 2 MB or 1 GB huge pages instead of the default 4 KB pages. Enable transparent huge pages on the host: echo always | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
Enable IOMMU: If your VPS provider exposes IOMMU, passing through PCI devices to nested VMs eliminates I/O virtualization overhead. Check with dmesg | grep -i iommu.
Tune CPU governor: Set the CPU governor to performance: echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
Use virtio drivers: Both disk and network devices in the nested VM should use virtio for near-native I/O performance. Add --disk bus=virtio --network model=virtio to your virt-install command.
Allocate CPU pinning: Pin each vCPU of the nested VM to a dedicated host pCPU to prevent CPU contention.

Step 6: Monitor Performance and Resource Usage

Monitor both the host VPS and the nested VMs to ensure you are not overprovisioning:

Host-level monitoring: Use htop for CPU/memory, iostat -x 2 for disk I/O, and virsh domstats vm-name for per-guest statistics.
Guest-level monitoring: Install netdata or prometheus-node-exporter inside each nested VM.
KVM-specific metrics: Run virt-top to see which VMs are consuming the most CPU, memory, and disk I/O.

If a nested VM shows high CPU steal time (visible with mpstat 1 inside the guest), you are overprovisioning the host vCPUs. Reduce the number of VMs or lower the vCPU count per VM.

Conclusion

Setting up a virtual machine on your VPS with KVM gives you maximum flexibility for testing, development, and multi-tenant workloads. Start with a single nested VM to validate performance, then scale up as your needs grow. Remember that nested virtualization works best on VPS plans with generous CPU allocations and NVMe storage. Compare VPS provider specs and benchmarks to find a plan that gives you enough headroom for nested VMs without breaking your budget.