VPS GPU Passthrough Guide: Setting Up CUDA and OpenCL Compute on Linux Servers

GPU compute on VPS instances unlocks machine learning training, scientific simulations, video transcoding, and rendering workloads that would be impractical on CPU-only infrastructure. While most cloud GPU solutions come from large providers, an increasing number of VPS hosts now offer GPU passthrough via PCIe passthrough (SR-IOV) or vGPU partitioning. This guide covers configuring NVIDIA CUDA and AMD OpenCL compute environments on Linux VPS instances with attached GPUs, including driver installation, containerization, and performance benchmarking.

GPU VPS Deployment Options

GPU access on a VPS typically comes in three flavors:

• PCIe Passthrough — A physical GPU is dedicated to a single VPS instance. Best performance, full memory access. Requires hardware support (IOMMU, VT-d/AMD-Vi).
• SR-IOV (Single Root I/O Virtualization) — A physical GPU is partitioned into multiple virtual functions (VFs), each assigned to a different VPS. Good isolation with near-native performance.
• vGPU (NVIDIA GRID / AMD MxGPU) — Time-sliced GPU sharing across multiple VPS instances. Suitable for light compute or virtual desktop workloads.

For CUDA/OpenCL compute, PCIe passthrough or SR-IOV is strongly recommended. When evaluating providers, compare VPS providers on our comparison table to identify those offering dedicated GPU passthrough with guaranteed VRAM allocation.

Prerequisites

• VPS with GPU passthrough enabled (NVIDIA Tesla T4, A10, A100, or AMD MI series)
• Ubuntu 22.04 or 24.04 LTS with kernel 6.2+
• IOMMU enabled in BIOS/hypervisor (check with dmesg | grep -i iommu)
• At least 16 GB system RAM and 50 GB NVMe storage
• NVIDIA driver version 550+ or AMD ROCm 6.0+

Step 1: Verify GPU Attachment

SSH into your GPU-enabled VPS and verify the hardware is visible:

# Check PCI devices for GPU
lspci | grep -E "VGA|3D|Display|NVIDIA|AMD"

# For NVIDIA, check after driver install
nvidia-smi

# For AMD, check after ROCm install
rocm-smi

# Verify IOMMU groups
for g in $(find /sys/kernel/iommu_groups/* -maxdepth 0 -type d | sort -V); do
  echo "IOMMU Group $(basename $g):"
  ls -1 $g/devices/
done

If no GPU appears in lspci, contact your provider to confirm GPU passthrough is enabled on the hypervisor.

Step 2: Install NVIDIA Drivers and CUDA Toolkit

For NVIDIA GPUs, use the official CUDA repository for the most stable deployment:

# Add NVIDIA CUDA repository
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update

# Install CUDA toolkit 12.5 (includes driver)
sudo apt install -y cuda-toolkit-12-5

# Reboot to load the NVIDIA kernel module
sudo reboot

After reboot, verify the installation:

nvidia-smi
# Expected output:
# +-----------------------------------------------------------------------------+
# | NVIDIA-SMI 550.xx    Driver Version: 550.xx    CUDA Version: 12.5          |
# +-----------------------------------------------------------------------------+

nvcc --version
# Should show Cuda compilation tools, release 12.5

Step 3: Install AMD ROCm and OpenCL (AMD GPUs)

For AMD GPUs, install the ROCm stack:

# Add AMD ROCm repository
wget https://repo.radeon.com/amdgpu-install/6.0/ubuntu/jammy/amdgpu-install_6.0.60002-1_all.deb
sudo dpkg -i amdgpu-install_6.0.60002-1_all.deb
sudo apt update

# Install ROCm runtime and OpenCL
sudo amdgpu-install -y --usecase=rocm --no-dkms

# Add user to render and video groups
sudo usermod -aG render,video $USER

# Reboot
sudo reboot

Verify ROCm:

rocm-smi
# Should list GPU temperature, power usage, and VRAM

clinfo | grep -E "Platform Name|Device Name|Driver Version"
# Should show AMD OpenCL platform and device

Step 4: Containerize GPU Workloads with Docker

Containerizing GPU workloads ensures reproducible environments across VPS instances:

# NVIDIA Container Toolkit
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt update
sudo apt install -y nvidia-container-toolkit
sudo systemctl restart docker

# Test GPU access in container
docker run --rm --gpus all nvidia/cuda:12.5.0-base-ubuntu24.04 nvidia-smi

For AMD GPUs with Docker:

# Use ROCm Docker image
docker run --rm --device=/dev/kfd --device=/dev/dri --group-add=render \
  rocm/dev-ubuntu-22.04:6.0 rocm-smi

Performance Benchmarks

Run these benchmarks to validate your GPU compute performance:

# CUDA bandwidth test (NVIDIA)
/usr/local/cuda-12.5/extras/demo_suite/bandwidthTest

# CUDA device query
/usr/local/cuda-12.5/extras/demo_suite/deviceQuery

# MLPerf mini benchmark (containerized)
docker run --rm --gpus all mlperf/mini:latest

# OpenCL benchmark (AMD)
sudo apt install clinfo clpeak
clpeak

GPU Model	CUDA Cores	VRAM	Memory Bandwidth	FP32 TFLOPS	Ideal For
NVIDIA T4	2,560	16 GB GDDR6	320 GB/s	8.1	Inference, small batch training
NVIDIA A10	9,216	24 GB GDDR6	600 GB/s	31.2	Training, rendering
NVIDIA A100	6,912	40/80 GB HBM2e	1.6 TB/s	19.5 (FP32) / 312 (TF32)	Large model training
AMD MI250	14,080 (stream)	128 GB HBM2e	3.2 TB/s	45.3 (FP32)	HPC, scientific computing

Common Issues and Troubleshooting

• GPU not detected in lspci — Confirm PCIe passthrough is enabled on the hypervisor. Some providers require a support ticket to attach a GPU.
• NVIDIA-SMI reports “No devices were found” — The NVIDIA kernel module may not have loaded. Run sudo modprobe nvidia and check dmesg | grep nvidia.
• CUDA out of memory — Monitor VRAM usage with nvidia-smi -l 1. Reduce batch sizes or use gradient checkpointing.
• IOMMU group issues on KVM hosts — Request PCIe ACS override from your provider if the GPU shares an IOMMU group with other devices.
• Docker GPU access denied — Ensure nvidia-container-toolkit is installed and Docker daemon was restarted. Verify the user is in the docker group.

Conclusion

GPU passthrough on a VPS brings data center-grade compute to a virtualized environment without sacrificing performance. With proper driver installation, containerization via Docker, and CUDA/OpenCL toolchain setup, you can run ML training, video transcoding, and HPC workloads on a VPS with near-bare-metal GPU performance. When choosing a provider for GPU compute, see performance benchmarks on our comparison page to compare GPU-equipped VPS plans based on VRAM allocation, GPU model availability, and pricing per compute hour.

Quick reference: Always pin your NVIDIA or ROCm driver version in production to avoid regression from automated kernel updates. Use nvidia-persistenced or rocm-persist to keep GPU state initialized between workloads.