TCP Fast Open on a VPS: How to Configure and Benchmark for Lower Latency

TCP Fast Open (TFO) is one of the simplest latency optimizations you can enable on a VPS. A single kernel parameter and an Nginx directive can eliminate one full round trip from every repeat TCP connection — reducing page load times by 40–50 ms for users on a 50 ms RTT connection. This tutorial explains how TFO works, how to configure it on your server, and how to benchmark the results to confirm the improvement.

How TCP Fast Open Works

A standard TCP connection requires a three-way handshake (SYN, SYN-ACK, ACK) before any application data can be sent. TFO modifies this by allowing the client to include data in the initial SYN packet — provided the client has a valid TFO cookie obtained from a previous connection. This collapses the handshake to zero extra round trips for repeat connections.

The TFO cookie is a cryptographic token generated by the server and sent to the client during the first connection. The client stores the cookie and presents it in subsequent SYN packets. The server validates the cookie, and if valid, processes the data immediately without waiting for the ACK to complete the handshake. This is defined in RFC 7413 and has been part of the Linux kernel since version 3.7.

Step 1: Enable TFO at the Kernel Level

The net.ipv4.tcp_fastopen sysctl accepts a bitmask:

• 0 — Disabled
• 1 — Enable for client (outgoing) connections
• 2 — Enable for server (incoming) connections
• 3 — Enable both client and server roles

For a web server, set value 3 to enable both roles:

# Enable TFO for both client and server roles
echo "net.ipv4.tcp_fastopen = 3" | sudo tee /etc/sysctl.d/99-tcp-fastopen.conf
sudo sysctl --system

# Verify
sysctl net.ipv4.tcp_fastopen
# Expected: net.ipv4.tcp_fastopen = 3

The server role (bit 2) is what allows incoming repeat connections to skip the handshake. The client role (bit 1) is useful if your VPS itself makes outbound connections — for example, to upstream APIs or external databases.

Step 2: Configure TFO in Nginx

Nginx exposes TFO via the fastopen parameter on the listen directive. The numeric value is the pending-SYN queue size for TFO connections:

server {
    listen 443 ssl http2 fastopen=256;
    listen 80 fastopen=256;
    server_name example.com;

    # ... rest of your Nginx configuration
}

The queue size should be proportional to the number of concurrent connections your server handles. For a low-traffic VPS, 256 is sufficient. For busy servers, values up to 1024 are reasonable. The queue only stores TFO connections that arrive before the three-way handshake completes, so it does not need to match the full connection backlog.

Test the configuration and reload Nginx:

sudo nginx -t
sudo systemctl reload nginx

Step 3: Configure TFO in Apache (Alternative)

If you use Apache instead of Nginx, enable TFO through the AcceptFilter directive:

# In /etc/apache2/apache2.conf or a vhost file
AcceptFilter https TCP_FASTOPEN
AcceptFilter http TCP_FASTOPEN

Restart Apache: sudo systemctl restart apache2.

Step 4: Verify TFO Is Working with Kernel Counters

Do not trust the configuration alone — verify with kernel counters using the nstat tool:

# Watch TFO counters in real time
watch -n 2 'nstat -az | grep -i fastopen'

Counter	Meaning	Expected Value
TCPFastOpenPassive	Inbound TFO connections accepted (server role)	Increases with traffic
TCPFastOpenActive	Outbound TFO connections (client role)	Increases if client role is enabled
TCPFastOpenFail	Cookie rejected or handshake failed	Should be near zero
TCPFastOpenListenOverflow	TFO dropped because accept queue was full	Should be near zero; increase fastopen queue size if growing

Generate test traffic and verify the counters increment:

# Reset counters
nstat -rs

# Generate TFO traffic from a client that supports it
curl --tcp-fastopen https://your-server.com/

# Check counters
nstat -az | grep FastOpen

Step 5: Benchmark the Latency Improvement

Use curl’s timing output to measure the difference TFO makes on a per-connection basis. Run this benchmark from a client machine (not the server itself, since you want to measure real network latency):

#!/bin/bash
# tfo-benchmark.sh - Compare with and without TFO
URL="https://your-server.com/"

echo "=== Without TFO ==="
for i in $(seq 1 10); do
  curl -s -o /dev/null -w "Request $i: %{time_total}s\n" "$URL"
done

echo ""
echo "=== With TFO ==="
for i in $(seq 1 10); do
  curl --tcp-fastopen -s -o /dev/null -w "Request $i: %{time_total}s\n" "$URL"
done

The first request in each series will be similar (no cookie yet for the TFO run). Starting from the second request, the TFO series should show a measurable reduction in time_total. On a VPS with 50 ms RTT to the client, expect a 40–50 ms reduction per request for repeat connections.

Step 6: Pair TFO with BBR for Maximum Effect

TFO reduces latency for new connections, but it does not affect throughput or congestion control. Pair it with BBR (Bottleneck Bandwidth and Round-trip propagation time) for a complete TCP optimization stack:

# Enable BBR congestion control
echo "net.core.default_qdisc = fq" | sudo tee /etc/sysctl.d/99-bbr.conf
echo "net.ipv4.tcp_congestion_control = bbr" | sudo tee -a /etc/sysctl.d/99-bbr.conf
sudo sysctl --system

# Verify
sysctl net.ipv4.tcp_congestion_control
# Expected: net.ipv4.tcp_congestion_control = bbr

BBR does not rely on packet loss as a congestion signal, making it ideal for virtualized environments where packet loss may be artificially induced by the hypervisor. Combined with TFO, you get lower latency for new connections and better throughput for established ones.

When TFO Does Not Help

TFO is not a silver bullet. It will not help in these scenarios:

• First-time visitors: TFO only benefits repeat connections. The first visit always requires a full handshake while the cookie is established.
• Long-lived keep-alive connections: HTTP/2 multiplexing, WebSockets, and persistent database connections reuse a single TCP connection, so TFO provides no benefit after the initial handshake.
• Behind a load balancer: Some cloud load balancers strip or modify TCP options, preventing TFO cookies from passing through.
• I/O-bound applications: TFO saves milliseconds. If your application spends 500 ms on backend processing, the TFO savings are negligible in the overall response time.

Troubleshooting

• TCPFastOpenFail is growing: Middleboxes (firewalls, NAT gateways) may drop SYN packets that carry data. The kernel detects this and retries without TFO automatically, so the failure mode is a single slow first attempt.
• TCPFastOpenListenOverflow is growing: Your fastopen queue size in Nginx is too small. Increase it from 256 to 512 or 1024.
• No TCPFastOpenPassive count: TFO is not enabled on the server side. Double-check sysctl net.ipv4.tcp_fastopen is set to 2 or 3.

TCP Fast Open is a single sysctl away from measurable latency savings on repeat connections. When choosing a VPS provider for latency-sensitive applications, compare VPS providers on our comparison table to check which ones offer the network infrastructure that makes these optimizations worthwhile. Combine TFO with BBR congestion control and proper TCP buffer tuning, and you can eliminate two round trips from every repeat HTTPS connection — a meaningful improvement for any latency-sensitive workload.