Setting Up Docker on a VPS: Step-by-Step with Compose, Resource Limits, and Security Hardening

Docker packages applications and their dependencies into lightweight, portable containers that run identically on any Linux server. Deploying Docker on a VPS gives you the isolation of virtual machines without the overhead, making it ideal for hosting multiple web apps, APIs, or databases on a single server. This guide covers installing Docker on Ubuntu 24.04, deploying multi-container applications with Docker Compose, enforcing resource limits, and hardening your setup for production. For VPS plans that can handle containerized workloads efficiently, check our VPS performance tuning guides.

Prerequisites: Minimum VPS Specs for Docker

Docker’s daemon uses ~100 MB RAM at idle, but your containers need memory too. For a typical setup hosting 3–5 containers:

Component	Minimum	Recommended
CPU	1 vCPU	2+ vCPUs
RAM	1 GB	4 GB
Storage	20 GB	40 GB NVMe
OS	Ubuntu 22.04+	Ubuntu 24.04 LTS
Docker Engine	27.x	27.x+ with Compose v2

Step 1: Install Docker Engine from Official Repos

Use the official Docker repository for the latest stable release. Do not use apt install docker.io — that version lags behind by months and may miss critical security patches:

# Remove old versions
for pkg in docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc; do sudo apt-get remove $pkg; done

# Add Docker's official GPG key and repository
sudo apt-get update
sudo apt-get install ca-certificates curl -y
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

# Install Docker packages
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y

Verify: sudo docker run hello-world. You should see the Hello from Docker! message.

Step 2: Post-Install — Run Docker Without Sudo

By default, Docker requires sudo. Add your user to the docker group:

sudo usermod -aG docker $USER
# Log out and back in (or run: newgrp docker)
docker run hello-world

Security warning: The docker group grants root-equivalent privileges. On multi-user VPS systems, use rootless Docker instead. See Docker’s rootless mode documentation.

Step 3: Multi-Container Apps with Docker Compose

Docker Compose defines multi-container applications in a YAML file. Below is a production-ready example running Nginx + PHP-FPM + MariaDB + Redis for a WordPress site, with resource limits and health checks built in:

# docker-compose.yml
services:
  db:
    image: mariadb:10.11
    restart: always
    volumes:
      - db_data:/var/lib/mysql
    environment:
      MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD}
      MYSQL_DATABASE: wordpress
      MYSQL_USER: wpuser
      MYSQL_PASSWORD: ${MYSQL_PASSWORD}
    deploy:
      resources:
        limits:
          cpus: '1.0'
          memory: 1024M
        reservations:
          cpus: '0.5'
          memory: 512M
    healthcheck:
      test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
      interval: 10s
      timeout: 5s
      retries: 3

  wordpress:
    image: wordpress:6-fpm-alpine
    restart: always
    depends_on:
      db:
        condition: service_healthy
    environment:
      WORDPRESS_DB_HOST: db:3306
      WORDPRESS_DB_USER: wpuser
      WORDPRESS_DB_PASSWORD: ${MYSQL_PASSWORD}
      WORDPRESS_DB_NAME: wordpress
    volumes:
      - wp_data:/var/www/html
    deploy:
      resources:
        limits:
          cpus: '0.5'
          memory: 512M

  nginx:
    image: nginx:alpine
    restart: always
    ports:
      - "8080:80"
    volumes:
      - wp_data:/var/www/html
      - ./nginx/default.conf:/etc/nginx/conf.d/default.conf:ro
    depends_on:
      - wordpress
    deploy:
      resources:
        limits:
          cpus: '0.25'
          memory: 128M

  redis:
    image: redis:7-alpine
    restart: always
    command: redis-server --appendonly yes --maxmemory 128mb --maxmemory-policy allkeys-lru
    volumes:
      - redis_data:/data
    deploy:
      resources:
        limits:
          memory: 256M

volumes:
  db_data:
  wp_data:
  redis_data:

Deploy with: docker compose up -d. Access WordPress at http://your-vps-ip:8080. For production, add a reverse proxy (Caddy or Traefik) to handle SSL termination on port 443.

Step 4: Container Resource Limits — Why They Matter

Without resource limits, a single runaway container can exhaust your VPS and take down every other service. Docker lets you constrain CPU, memory, and I/O per container:

# Hard memory limit (container is killed if exceeded)
docker run --memory=512m nginx

# Soft reservation (container guaranteed at least 256 MB)
docker run --memory=512m --memory-reservation=256m nginx

# CPU limits
docker run --cpus=0.5 nginx              # 50% of one core
docker run --cpus=2 --cpuset-cpus=0,1 nginx  # exactly cores 0 and 1

# Block I/O limits
docker run --device-read-bps /dev/nvme0n1:10mb --device-write-bps /dev/nvme0n1:5mb nginx

In Docker Compose, set these under the deploy.resources.limits key as shown in Step 3. On a 4 GB VPS running five containers, resource limits prevent a memory leak in one container from OOM-killing the entire server.

Step 5: Security Best Practices for Docker on VPS

• Never expose the Docker socket (/var/run/docker.sock) inside a container unless absolutely required — it grants full host control. Use Docker’s API proxy or socket proxy containers instead.
• Use read-only root filesystems for containers that don’t need write access: docker run --read-only --tmpfs /tmp nginx.
• Run containers as non-root with the USER directive in your Dockerfile. Use --user flag at runtime: docker run --user 1000:1000 nginx.
• Keep base images updated: regularly run docker pull and rebuild. Subscribe to Docker Hub advisory feeds for critical CVEs.
• Scan images for vulnerabilities: docker scout quickview your-image (included with Docker Desktop and Docker Engine 27+).
• Enable Content Trust: export DOCKER_CONTENT_TRUST=1 ensures only signed images can be pulled and run.
• Restrict network capabilities: use --cap-drop=ALL --cap-add=NET_BIND_SERVICE to drop all capabilities and only add back what’s needed.

Step 6: Log Rotation and Monitoring

Container logs accumulate quickly. Configure global log rotation in /etc/docker/daemon.json:

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}

Restart Docker: sudo systemctl restart docker. This limits each container to three 10 MB log files (30 MB max per container). For centralized logging, configure the fluentd or gelf log drivers to ship logs to an external aggregator.

Troubleshooting Common Docker on VPS Issues

• “No space left on device”: Clean up unused images with docker system prune -a. Check overlay filesystem usage with docker system df.
• DNS resolution fails inside containers: Set --dns 8.8.8.8 --dns 1.1.1.1 or configure /etc/docker/daemon.json with a DNS section.
• Timeouts pulling images: On slower VPS connections, increase Docker’s pull timeout: add "max-concurrent-downloads": 3 to daemon.json.
• iptables conflicts: If your VPS has a firewall (UFW, firewalld), Docker’s iptables rules may conflict. Set "iptables": false in daemon.json and manage firewall rules manually.

Docker transforms a general-purpose VPS into a flexible application platform. Once you have the basics running with Compose, resource limits, and security hardening, explore Docker Swarm for multi-node orchestration or Kubernetes for larger deployments. For now, these patterns cover the vast majority of single-server production setups. For VPS plans with the CPU and memory headroom to run containerized applications smoothly, check our VPS performance tuning guides.