How Redis persists data: RDB, AOF and the BGSAVE fork

The Redis hot path never touches disk. Every GET walks an in-memory dict; every SET mutates an in-memory value and returns. This is why a single Redis process handles hundreds of thousands of ops per second on a modest box. Persistence happens alongside, so that a power cut doesn't erase the dataset.

There are four configurations. Nothing suits a cache whose dataset can be regenerated from Postgres or DynamoDB, where losing all keys just means a cold cache that refills on miss. Plenty of production Redis deployments persist nothing. RDB only: periodic binary snapshots; lose up to the snapshot interval on crash. AOF only: append every write to a command log; lose up to one fsync window. Both, the usual recommendation since 4.0: RDB for fast restart, AOF for crash durability.

Each has its own failure profile. The question to ask is how much data you can afford to lose if the kernel panics right now, and each configuration gives a different answer. The rest of this page covers the mechanics behind those answers.

The naive way to snapshot the keyspace is SAVE: the main thread walks the dict, serialises every key, writes the RDB file. While it does this the server is unresponsive, because Redis is single-threaded. On a 10 GB instance that takes tens of seconds, so SAVE is rarely used in production.

BGSAVE forks. The child process inherits the parent's address space and writes the snapshot from the inherited memory. The parent keeps serving commands. This is cheap because of copy-on-write: after the fork, both processes share the same physical pages, marked read-only by the kernel. The parent only pays for a page when it writes to it, at which point the kernel duplicates that 4 KB page and gives the parent a private copy. If the parent isn't writing much during the snapshot, the COW cost is close to zero.

If the parent is on a write-heavy workload during BGSAVE, COW cost approaches duplicating the entire keyspace. This causes the well-known Redis fork latency spike: a 30 GB instance can stall for 100 ms to 1 s during the fork itself (the page-table copy), then use extra memory for the rest of the snapshot. Antirez wrote about this at length. The mitigations are simple: schedule BGSAVE during low-write windows; disable Transparent Huge Pages on the host (2 MB pages make COW 512× more expensive per copy); run a dedicated replica that does the persistence so the master never forks.

The save 3600 1 300 100 60 10000 default triggers BGSAVE automatically: every hour if at least one key changed, every five minutes if 100 changed, and every minute if 10,000 changed. Tune it to your write rate and acceptable loss window.

The AOF (append-only file) records every write command in the same RESP wire format the clients sent. SET user:1 Ada goes into the keyspace and into the AOF. On restart, Redis opens the AOF and replays it from the beginning, reconstructing the in-memory state by re-running each command. There is no separate recovery algorithm, only deterministic replay of the write stream.

The main setting is appendfsync, which has three values that trade durability for throughput:

always

fsync after every command. Sub-millisecond loss window. Throughput drops to roughly 30-50% of everysec because every write pays the fsync cost (typically 0.5-2 ms on an enterprise SSD with a supercap, and worse on consumer drives). Use it when the data cannot be recovered elsewhere and losing a single write counts as a bug.

everysec

Default. A background thread fsyncs the buffered writes once per second. Worst-case loss: about one second of writes on crash. Throughput is close to running without AOF, because the fsync is off the hot path. It suits most production deployments.

no

Never call fsync. The OS flushes when it chooses, usually every 30 s on Linux, so the worst-case loss is whatever sits in the page cache, which can be tens of seconds of writes. This is the fastest setting and fits cases where the AOF exists but durability doesn't matter.

The background fsync thread is one of the few places where Redis departs from strict single-threading: the main thread queues bytes and the AOF thread flushes them. If the AOF thread is stuck because the disk is slow, the main thread eventually blocks as the AOF buffer grows, which shows up as p99 latency spikes. Check INFO persistence for aof_delayed_fsync to catch this.

The AOF grows without bound by default. A counter that increments once per request produces one AOF entry per request, which can reach terabytes per day at scale even though the counter in memory is a single integer. Without compaction the AOF would fill the disk.

BGREWRITEAOF does the compaction. Redis forks a child (same fork trick, same COW behaviour, same caveats), and the child walks the current keyspace, writing a fresh AOF that reproduces the same final state with the minimum number of commands. A million increments collapse to one SET counter 1000000. The new file is typically a small fraction of the old one.

While the child is writing, the parent is still taking writes. Those go into both the live AOF buffer and a separate AOF rewrite buffer. When the child finishes its fresh AOF, the parent appends the rewrite-buffer contents, atomically renames the new file over the old one and deletes the old AOF. Clients don't see the handoff. By default a rewrite triggers at auto-aof-rewrite-percentage 100, when the AOF has doubled in size since the last rewrite.

Redis 7.0 changed the on-disk shape: multi-part AOF. Instead of one growing file, the AOF is now a base file (an RDB-format snapshot by default, controlled by aof-use-rdb-preamble) plus a sequence of incremental AOF files for commands after the base. This layout makes the "RDB plus AOF tail" recovery path explicit on disk instead of implicit in one file header. The Redis 7.0 release notes have the migration details.

A Redis replica receives the master's write stream and applies it in order. If the master crashes, you fail over to a replica. This protects against machine failure as well as process failure.

Two caveats matter for persistence. The first comes from the Jepsen analyses (Aphyr, 2013 and 2014): Sentinel-managed failover during a network partition can elect a replica that's behind on commands, and writes the old master had accepted but not yet replicated are silently lost on failover. Redis added min-replicas-to-write as a mitigation (refuse writes if fewer than N replicas are connected), but it trades availability for durability and does not fully fix the problem. Redis does not provide strong consistency, so if you need that, use RedisRaft or a different system.

The second is an operational pattern. Replicas can have separate persistence settings: a master with AOF and even RDB disabled, and replicas with AOF=everysec and periodic BGSAVE. The master never forks or fsyncs, so it never pays that latency, and the replicas do the disk work. Writes reach the replicas' RAM quickly (sub-millisecond on a 10 Gb network, though replication is asynchronous) and reach their disks later, off the critical path. Large deployments such as Twitter's commonly used this layout.

Redis 7 brought the multi-part AOF along with smaller persistence improvements such as better fsync error reporting and smarter rewrite triggers. RedisRaft, developed for years at Redis Labs, would bring Raft-based strong consistency to a Redis cluster and replace best-effort failover with guarantees like those of etcd or Consul. It is not generally available as of this writing, but the design is on GitHub.

Redis Enterprise's Redis on Flash uses a different model. Hot keys stay in RAM, while cold keys and the values of less-active keys live on SSD with only metadata in RAM. A 10 TB dataset on a host with 64 GB of RAM, served at single-digit-millisecond latency for the hot path and 100-500 µs for the cold path. Persistence changes as a result: the SSD layer is the durable layer and the RAM tier acts as a cache. RDB and AOF still exist for the metadata, but the keyspace itself sits on persistent storage.

At larger scale, tuning RDB and AOF is only part of it. Pick a per-shard persistence mode that matches the per-shard write rate, do the persistence work on replicas so the master never forks, shard before fork latency starts to hurt, and run multi-AZ replicas with semi-sync writes if the application needs that durability. RDB and AOF remain the primitives, and the work is in how they combine with sharding and replication.