Redis pipelining and throughput simulator

Each client sends a batch of commands (one, without pipelining), waits until all the replies are back, then sends the next. The server works on one batch at a time: a fixed cost for reading the request and writing the replies, and a small cost per command. Change the round-trip time, number of clients and pipeline depth, and the panel shows which limit you are hitting: each client’s round trips or the server’s CPU. The default costs, 1 µs per command and 9 µs of socket I/O per batch, are illustrative; they make pipelining’s best case 10 times the unpipelined rate, the figure the Redis documentation gives.

Redis answers simple commands in microseconds, but a client that sends one command and waits for the reply before sending the next can do at most one command per network round trip. The Redis documentation’s example is a slow 250 ms link, which allows at most 4 commands per second however fast the server is; the simulation gives 3.98. Even in one data centre, with a round trip of a fraction of a millisecond, a single waiting client stays far below what the server could do: with a 0.2 ms round trip, one client managed 4,761 commands per second unpipelined and 71,104 with batches of 16.

Pipelining sends many commands before reading any replies. It removes the round trip per command, and it also helps the server: according to the Redis documentation, many pipelined commands are read with a single read() system call and their replies sent with a single write(), and since the switch into the kernel costs far more than executing a simple command, throughput grows almost linearly with pipeline length at first and eventually reaches about 10 times the unpipelined rate. With 50 clients and a 0.2 ms round trip:

Redis serves every request from one thread; its documentation calls the design mostly single-threaded, since some slow I/O happens in background threads. Commands run one after another, which makes each one atomic without locks, but as the documentation puts it, when a request is slow to serve, all the other clients wait for it. Commands that touch the whole dataset or a huge value, like KEYS or an LRANGE over a million elements, run for milliseconds or longer while everyone else waits. In the simulation, one 20 ms command every 100 ms took the median latency from 1.25 ms to 1.25 ms and the 99th percentile from 1.25 ms to 21.25 ms. The Redis latency guide recommends finding such commands with the slow log, says KEYS should only be used for debugging, and points to the incremental SCAN family instead.

A pipeline is only a network optimisation: the server may run other clients’ commands between the commands of one pipeline. When commands must run together, MULTI/EXEC transactions and Lua scripts execute them as a unit that nothing else interleaves with. Both can be sent pipelined. Multi-key commands such as MGET and MSET are a third way to batch, one command doing the work of many.