WebSockets, a two-way connection that stays open over HTTP

WebSockets are a bidirectional, full-duplex communication channel over a single TCP connection. RFC 6455 (2011) standardised them. The connection starts as HTTP and upgrades to WebSocket via the Upgrade header; once upgraded, both client and server can send messages at any time. Modern realtime apps (chat, collaboration tools, live dashboards) use WebSockets, with HTTP/3 and Server-Sent Events as alternatives for one-way pushes.

For years, HTTP was strictly half-duplex: the browser asked and the server answered. If the server had news (a chat message, a stock tick, the other player’s move) it had no way to say so until the client asked.

The workarounds got increasingly awkward. Polling hits the server every second and wastes most of the requests. Long polling holds a request open and answers it when something happens, after which the client re-opens it immediately. Hidden iframes stream an infinite HTML document and parse <script> tags as they arrive. Each traded bandwidth, latency or complexity for a server-initiated message.

WebSockets replaced these workarounds. RFC 6455, published in December 2011, defines a way to start a connection as HTTP, switch protocols mid-request, and keep the underlying TCP socket open for as long as both sides want. After the handshake, either side can send at any time. The request/response shape no longer applies.

The handshake is deliberate. WebSockets use ports 80 and 443 and start as HTTP so that corporate firewalls that allow "web traffic" also allow them. The protocol could have defined its own port, as FTP (21) and SSH (22) did, but a protocol that firewalls block is hard to deploy.

Three headers do the conversion. Connection: Upgrade and Upgrade: websocket signal the protocol switch. Sec-WebSocket-Key is a random 16-byte nonce, base64-encoded, and the server uses it to prove it understands WebSockets.

The simulator below plays the whole lifecycle: the upgrade, a few data frames in both directions, a keepalive pair and the close. Use the step controls to move through it. Click Show wire bytes to see the frame layout on the wire.

Every WebSocket message is one or more frames. The frame header holds almost all of the protocol’s logic (opcodes, length, masking, fragmentation) in as few as two bytes. HTTP headers, by comparison, routinely reach 800 bytes.

Reading the header from the top: FIN is a single bit: set it when this is the last frame of a message (so you can split big messages across frames without them interleaving with other messages). RSV1-3 are reserved for extensions; RSV1 carries permessage-deflate compression when negotiated. Opcode is four bits, which we spell out below. MASK is a single bit: always 1 from client to server, always 0 from server to client. Payload length is seven bits, with sentinel values that signal a following 16-bit or 64-bit length. Short messages therefore pay for only seven bits of length, while long messages can still express up to eight exabytes.

The four-bit opcode field has sixteen possible values; RFC 6455 assigns six and reserves the rest. Three carry application payloads (TEXT, BINARY, CONTINUATION). Three are control frames sent by the protocol itself (CLOSE, PING, PONG). Control frames must fit in a single unfragmented frame of ≤125 bytes, so they are never part of a streaming message.

Every frame from the client carries a 32-bit masking key. Every byte of the payload is XORed with a rotating byte of that key before it hits the wire. The server reverses the XOR on receipt. The net effect on the payload is nil, so why bother?

The reason is cache-poisoning intermediaries. An attacker controlling a browser via JavaScript could try to craft a WebSocket payload whose bytes look like a valid HTTP GET request to a transparent proxy sitting between the client and the server. The proxy, not knowing WebSockets, might interpret those bytes as a new request, fetch the response, and cache it under a URL the attacker chose. Later users get the poisoned cache entry.

Masking makes this attack impractical. The attacker controls the plaintext but not the random key, so the bytes that traverse the intermediary are unpredictable per frame, and a proxy that decodes them as HTTP sees gibberish. Server-to-client frames are never masked because there are no dangerous intermediaries in that direction: the server does not live behind the victim’s corporate proxy.

A stateless HTTP API is easy to scale. Put fifty machines behind a load balancer; any request goes to any machine; if one dies, another answers in milliseconds. WebSockets break each of those assumptions.

Because the socket lives for minutes or hours, a user stays tied to one server, call it Server A, for the whole session. The load balancer cannot round-robin individual frames. If Server A restarts, every connected user is disconnected and has to reconnect, possibly to a different server. The state that server held (room membership, typing indicators, subscription filters) is gone unless it was written somewhere durable.

This creates two scaling problems. The first is the C10K problem: how many idle sockets can one box hold? In 1999 the answer was about ten thousand, limited by the threads-per-connection model. Modern runtimes (Node, Go, Rust’s Tokio, Java’s Netty) hold each socket as a few hundred bytes of epoll state, and a single machine can comfortably serve a million. The second is fan-out: when a message arrives for a user connected to Server A, and someone else typing on Server B wants to send it, how does it get there?

The handshake is a plain HTTP GET. Browsers attach cookies to HTTP GETs whether or not the request is same-origin, unless SameSite says otherwise. So a malicious page at evil.example can execute new WebSocket("wss://yourbank.com/api") in a background script. The browser sends the user’s authenticated session cookie with the upgrade request, the bank’s server accepts it, and the attacker has a live, authenticated WebSocket to someone else’s bank session.

This is Cross-Site WebSocket Hijacking, CSWSH, and it is the WebSocket counterpart of CSRF. CORS does not apply to WebSockets, so the browser will not block the upgrade on an Origin mismatch the way it would block a cross-origin fetch. The defense has to live on the server.

One node holds many sockets. Modern Linux easily handles 1M+ idle TCP sockets per box (with ulimits and sysctl tuned). The harder problem is the application, since each socket holds buffer memory, inbound frames need processing, and per-connection ping/pong uses CPU. Reported numbers: Phoenix LiveView (Elixir) reports ~2M concurrent connections per node; Go services ~1M; Node.js ~300-500k.

Load-balancer stickiness. WebSockets are long-lived and stateful: the same client must keep hitting the same server. L7 load balancers (NGINX, Envoy, HAProxy) handle this with cookie-based or hash-based affinity. AWS ALB and GCP LB both support WebSocket out of the box; sticky sessions need to be enabled explicitly.

Reconnect storms. A node failure or deploy causes every connected client to reconnect at once. Without staggered backoff, the surviving nodes get a stampede. Production clients use exponential backoff with jitter (for example 1 s, then 2 s, then 4 s, each with ±50% jitter). Slack’s Flannel and Discord’s gateway both document stagger strategies.

Cross-node fan-out. When user A on node 1 sends a message to user B on node 17, the application needs a pub/sub layer. Common patterns: Redis Pub/Sub (simplest, lossy under load), Kafka (durable, higher latency), NATS (low-latency in-memory pub/sub). Choose by whether messages must survive a node restart.