Running an MQTT Broker on a VPS: Mosquitto Capacity, Persistence, and Connection Limits

Mosquitto handles a surprising number of MQTT connections for a small process, but the limit that bites operators is rarely the broker itself. It is file descriptors, TLS handshake CPU, or a persistence setting that turns a 5,000-message-per-second stream into a disk-bound crawl. Sizing a broker properly means working out three numbers: memory per connection, cost per persisted message, and the CPU budget for TLS.

The figures below come from Mosquitto 2.0.18 on a 2 vCPU / 4 GB instance running Debian 12, with clients connecting over TLS and a mix of QoS 0, 1, and 2 traffic. For instance sizing that matches these loads, check the VPS tiers before you start.

Connection-count memory math

An idle MQTT connection over TCP costs Mosquitto roughly 2 to 3 KB, plus kernel socket buffers of about 4 KB each way once data flows, so budget 10 KB per connection to be safe. Ten thousand idle connections therefore need around 100 MB, not 10 GB. The base process uses 8 to 12 MB before any client connects.

Active connections change the picture. Each in-flight QoS 1 or 2 message needs a copy of the payload for every subscriber, plus a 2 KB fixed per-message overhead in the broker’s queues. A 1 KB payload fanned out to 50 subscribers is 50 KB per publish, transient but real. With max_inflight_messages at its default of 20, a single slow client can hold 20 queued messages; ten slow clients hold 200. Cap queue depth and you cap memory.

What QoS 1 and QoS 2 actually cost to persist

With persistence disabled, QoS 1 publishes live only in memory and cost less than 100 microseconds of broker CPU each. With persistence true, Mosquitto writes sessions and retained messages to mosquitto.db, not every message. Retained messages are the expensive part: 100,000 retained topics at 200 bytes average grew the database file to about 60 MB and added 2.5 seconds to startup while it loaded.

QoS 2 adds a four-part handshake per message, so CPU per publish roughly doubles against QoS 1 and latency rises with round trips. For durable delivery, QoS 1 plus a persistent session (clean_session false, or MQTT 5 session-expiry-interval) gives almost the same guarantee with less overhead. Reserve QoS 2 for billing or command messages where duplicates are genuinely unacceptable, and measure: we saw 9,000 messages per second at QoS 1 drop to 5,200 at QoS 2 on the same box.

max_connections, max_inflight_messages, and queue limits

• max_connections defaults to unlimited, which means the OS limit becomes the real cap. Set it explicitly to a number you have sized for, such as 5000.
• max_inflight_messages defaults to 20; lower it to 8 to 10 for constrained clients and memory predictability.
• max_queued_messages defaults to 1000 per client; at 1 KB average that is 1 MB per stalled client, so 500 stalled clients would need 500 MB.
• max_packet_size defaults to 256 MB; set it to 256 KB or 1 MB unless you send large images, to stop a single client exhausting memory.
• message_size_limit and max_keepalive bound the remaining edges.

These five settings together determine your worst case. Calculate it as connections times queued messages times payload size, and keep the result under half of available RAM. Then verify with mosquitto_sub -t '$SYS/#' -v, which exposes $SYS/broker/clients/connected and $SYS/broker/messages/stored.

File descriptors and kernel sysctl limits

Every client needs one file descriptor, and if you raise max_connections without raising limits the broker will fail to accept new sockets with a vague error. systemd’s default LimitNOFILE is often 1024. Set LimitNOFILE=65536 in the unit file, and confirm with cat /proc/$(pidof mosquitto)/limits.

On the kernel side, raise net.core.somaxconn to 4096, net.ipv4.tcp_max_syn_backlog to 8192, and fs.file-max to 2097152. For brokers with tens of thousands of mostly idle connections, net.ipv4.tcp_keepalive_time at 300 seconds clears dead sockets faster than the minute-long MQTT keepalive on its own. Write these into /etc/sysctl.d/99-mosquitto.conf and reload, not into a shell one-liner you will forget.

TLS handshake CPU cost

A full RSA-2048 handshake costs about 1 ms on a modern core; ECDSA P-256 cuts that to roughly a third and halves the bytes exchanged. That sounds trivial until 2,000 devices reconnect simultaneously after a power cut, which is 2 seconds of pure handshake work plus the connection churn on top. Use ECDSA certificates, enable TLS 1.3 to benefit from session resumption, and set a session ticket lifetime of several hours so reconnects skip the expensive path.

On 2 vCPU, expect about 4,500 new TLS connections per second in a handshake-only benchmark, dropping below 2,000 once message traffic shares the cores. If your fleet is larger than a few thousand devices, put a TLS-terminating proxy or a load balancer in front, or accept longer reconnect storms. per_listener_settings true lets you keep a plaintext listener on localhost for bridges and monitoring without TLS overhead.

Bridging to a second broker

Bridges replicate topics between brokers, typically from an edge node to a central one. Configure them in mosquitto.conf with connection, address, and topic lines, setting bridge_protocol_version mqttv311 unless both ends speak MQTT 5. Use try_private false when bridging to non-Mosquitto brokers such as HiveMQ or EMQX, or the bridge fails silently.

Two settings matter for throughput. notifications_local_only true stops looped bridge messages from bouncing between nodes. bridge_queue_qos and the per-bridge queue length, default 64 messages, determine how much traffic survives a WAN blip; raise it to 1000 if your link is flaky, but remember every queued message is memory. Restrict bridge topics to narrow prefixes rather than #, because copying all traffic to a second site doubles bandwidth and persistence load for no benefit.

Monitoring what actually breaks first

The four metrics that predict failure are connected clients, messages stored, heap or RSS growth, and file descriptors in use. Watch $SYS/broker/clients/connected, $SYS/broker/messages/stored, $SYS/broker/load/messages/received/1min, and the process RSS from node_exporter. A steady climb in messages stored means clients are not acknowledging and queues are filling; that is your first sign of a network or client bug, not a broker bug.

Alert on three conditions: RSS above 70 percent of the instance, file descriptors above 60 percent of LimitNOFILE, and any drop in connected clients greater than 10 percent in a minute. For persistence, watch mosquitto.db growth; if it exceeds a few hundred megabytes, prune retained topics with mosquitto_sub -t '$SYS/broker/retained messages/count' and a null-payload publish. Choose an instance with headroom for a reconnect storm, as described on the hosting plans page, and size for twice your steady client count.

A workable starting configuration

For a broker expecting up to 5,000 devices: max_connections 8000, max_inflight_messages 10, max_queued_messages 200, max_packet_size 262144, TLS 1.3 with ECDSA certificates, LimitNOFILE=65536, persistence enabled only if retained state matters. That profile ran at 180 MB RSS with 5,000 idle TLS connections on a 2 GB instance and left room for a monitoring agent.

Then test before trusting it. Use mqtt-benchmark or EMQX’s emqtt_bench to simulate your real payload mix at twice the expected rate, and run it for an hour so queues and persistence have time to reveal themselves. If the broker is a core part of your product, verify the provider’s CPU is dedicated on the VPS comparison pages, because a shared core at 20 percent steal will show up as message latency long before it shows up as an error.