Redis persistence: how much data do you lose on restart or kill -9? Intermediate

The problem

A Redis 7.4 instance holds a product cache (reloadable from PostgreSQL), login sessions, and per-IP rate limit counters. Everything lives in RAM; persistence is how Redis writes the dataset to disk so it can reload it on startup.

When the container is kill -9ed or the machine loses power: losing the cache only raises database load, losing sessions logs users out, and losing counters gives an IP an extra window of requests.

The question: with the current configuration, what is left after a Redis crash, and what is the most you can lose?

What to choose? (summary first)

Situation	Choose	Condition
Pure cache	Disable persistence (save "", appendonly no) or sparse RDB	The database can handle a cold start; with replicas, the master must not auto-restart
Sessions, rate limit counters	AOF everysec + keep RDB	Accept losing about 1 s of writes when the machine crashes
Data that must not be lost	Do not make Redis the only source of truth; if unavoidable: AOF always + replication + WAIT	Throughput drops sharply; failover can still lose acknowledged writes

The Redis docs recommend enabling both when you need safety close to PostgreSQL’s; RDB is still useful for backups and fast restarts.

Core concepts

	RDB	AOF
What it writes	A snapshot of the whole dataset (dump.rdb)	A log of every write command, replayed on startup
When	save <seconds> <changes> rules or BGSAVE	Every write command; fsync per appendfsync
Lost on crash	Every write after the last snapshot	Depends on appendfsync (table below)
Default in 7.4	On: save 3600 1 300 100 60 10000	Off: appendonly no

The default save rules: snapshot after 3600 s if ≥ 1 change, after 300 s if ≥ 100, after 60 s if ≥ 10000. Check with CONFIG GET save.

fsync forces the kernel to push data from the page cache to disk. Data that has been write()n but not fsynced survives the process dying, but is lost when the machine crashes.

appendfsync	Behavior (per the docs)	Lost when the machine crashes
always	fsync before replying to the client, grouping several commands into one fsync	Almost nothing; very slow
everysec (default)	fsync every second on a background thread	About 1 s; up to 2 s if a slow fsync makes Redis delay write()
no	The kernel flushes on its own (Linux typically about 30 s)	Depends on the kernel

How it works

Session B and an INCR command pass through RAM, dump.rdb, and the AOF directory, then Redis dies:

After a crash, RDB rolls back to the last snapshot; AOF everysec only loses what was not yet fsynced

◀ Prev▶ PlayNext ▶↺ RestartStep 1/7

The latest snapshot contains session A. From here on, RDB stays frozen until the next save.

Point	Memory	dump.rdb	AOF on disk
Snapshot	A	A	A
SET B, fsync	A, B	A	A, B
INCR	A, B, rl=1	A	INCR not yet fsynced
Restart		RDB only: A remains	A, B; INCR survives if only the process died

RDB: fork and copy-on-write

BGSAVE calls fork(). The child writes the snapshot to a temp file and then renames it over the old file, so dump.rdb is never half-written; the parent keeps serving clients.

Copy-on-write: after the fork, parent and child share memory pages; the kernel copies a page only when the parent modifies it. The more writes during the save, the more RAM grows. Two things to measure:

• Fork latency: fork() copies the page table on the main thread. See latest_fork_usec (INFO stats); the docs cite roughly 9–13 ms/GB on modern hardware.
• CoW memory: the Fork CoW for RDB: ... log line and rdb_last_cow_size in INFO persistence. The docs require disabling transparent huge pages, because THP causes nearly all memory to be copied.

AOF: rewrite and multi-part since Redis 7

100 INCRs produce 100 entries for one key. A rewrite rebuilds the minimal command set: run via BGREWRITEAOF or automatically when the AOF doubles in size since the last rewrite and is ≥ 64 MB (auto-aof-rewrite-percentage 100, auto-aof-rewrite-min-size 64mb).

Since 7.0, the AOF is the appenddirname directory (default appendonlydir):

• base: the snapshot at rewrite time; in RDB format (.base.rdb) because aof-use-rdb-preamble yes is the default.
• incr: write commands after the base (.incr.aof).
• manifest: the list of files and their load order.

During a rewrite, the parent keeps writing to a new incr file, the child writes a new base, then Redis swaps the manifest atomically; if the rewrite fails, the old files still hold all the data.

Which file a restart loads

Loading diagram…

The docs: when both are enabled, the AOF is loaded because it is the most complete. The “empty AOF” branch was reproduced in the demo.

Operational details

• Clean shutdown (SIGTERM): with save rules, Redis writes an RDB before exiting (demo: docker stop kept all 5000/5000 keys). docker stop only waits 10 s before SIGKILL.
• stop-writes-on-bgsave-error yes (default): if BGSAVE fails, for example on a full disk, Redis refuses writes.
• aof-load-truncated yes (default): an AOF truncated at the end still loads; corruption in the middle needs redis-check-aof.

Replication does not replace persistence

The replication docs describe it: a master with persistence off crashes and is auto-restarted with an empty dataset; the replicas resync and wipe their copies too. With Sentinel, the master can restart so fast that the failure goes undetected.

Recommendation: enable persistence on both master and replicas; if you can’t, disable auto-restart for the master. Replication is asynchronous; WAIT only lowers the probability of losing writes.

Real-world example

Redis holding sessions and rate limit counters

Default configuration (RDB only). The container is kill -9ed at 10:07, with the last snapshot at 10:02: 5 minutes of sessions disappear. The team enables AOF everysec via CONFIG SET and keeps the save rules as a backup.

Demo: a dedicated Redis 7.4.9 (port 6391), writing batches of 50 SETs continuously for 3 seconds, docker kill right after the last acknowledged batch, restart, count DBSIZE:

npx --prefix examples ts-node examples/src/redis-persistence/crash-demo.ts rdb   # also everysec, always, none

Configuration	Client received OK	Left after restart	Lost
No persistence	589,800	0	589,800
RDB save 60 1000, snapshot at second 60	1,058,300	411,550	646,750
AOF everysec	630,950	630,950	0
AOF always	183,850	183,850	0

Reading the results:

• docker kill only kills the process; data already write()n is still in the page cache, so everysec loses 0. Only a power loss exposes the unfsynced part; the demo cannot simulate that.
• always got about 3.4 times fewer OKs than everysec (Docker Desktop on macOS, relative comparison only; numbers vary per run).

Real output: the AOF directory, enabling AOF the wrong way, fork and CoW

[everysec] after restart: 630950 keys remain, 0 keys lost
-rw------- 1 redis redis 22183987 appendonly.aof.2.base.rdb
-rw------- 1 redis redis 33363023 appendonly.aof.2.incr.aof
-rw------- 1 redis redis       88 appendonly.aof.manifest
file appendonly.aof.2.base.rdb seq 2 type b
file appendonly.aof.2.incr.aof seq 2 type i

Restart with --appendonly yes on a volume holding a 5000-key dump.rdb:

* Creating AOF base file appendonly.aof.1.base.rdb on server start
DBSIZE → 0

A 383 MB dataset, BGSAVE while redis-benchmark writes random SETs:

latest_fork_usec:2475
Fork CoW for RDB: current 108 MB, peak 108 MB, average 72 MB

Trade-offs

What persistence gives you

• RDB: compact file, fast restart, safe to copy for backup while Redis is running
• AOF everysec: almost no writes lost when the process dies; about 1 s lost when the machine crashes
• Multi-part AOF: rewrites don’t buffer writes in RAM, and an RDB-format base loads fast

The price you pay

• Fork blocks the main thread; CoW needs spare RAM
• AOF always cuts throughput several times over
• A slow fsync can cause latency spikes

Production checklist

1. Separate pure-cache instances from instances holding sessions.
2. Sessions, counters: appendonly yes, appendfsync everysec, keep the save rules; verify with CONFIG GET after deploys.
3. Reserve RAM for CoW, disable transparent huge pages, alert on latest_fork_usec and rdb_last_cow_size.
4. Alert when rdb_last_bgsave_status, aof_last_bgrewrite_status, or aof_last_write_status is not ok.
5. Switch from RDB to AOF with CONFIG SET appendonly yes, wait for the rewrite to finish, CONFIG REWRITE, and only then restart.
6. With replicas: enable persistence on the master or disable its auto-restart.
7. Shutdown timeouts (docker stop -t, terminationGracePeriodSeconds) long enough for the final save.
8. Copy the RDB or the AOF directory off the machine regularly and test restores.

Common mistakes

Enabling AOF by editing the config and restarting

Redis sees appendonly yes but no appendonlydir, so it creates an empty AOF and ignores dump.rdb: in the demo, 0 keys remain after the restart. Enable it with CONFIG SET appendonly yes on the running server, wait for aof_rewrite_in_progress:0, then update the config file.

The second trap is pairing replication with a master that has no persistence:

With replicas, the master doesn't need persistence

A master without persistence is restarted by the supervisor and comes up empty; the replicas sync to it and wipe all their data. Enable persistence on the master or forbid auto-restart.

Misconception	Consequence	What to do
With both enabled, Redis loads the newer file	Restores the wrong data	With AOF present, Redis always loads the AOF
always means nothing is ever lost	Still lost on failover to a replica that hadn’t received the write	Keep the source of truth in a transactional database

Self-check

1. RDB only, default rules, 50 changes written at 10:00, kill -9 at 10:04. What remains after restart?
2. Why does docker kill with everysec lose 0 keys, while a power loss can lose about 1 second?
3. A 20 GB dataset with heavy writes during BGSAVE: which metric tells you how much spare RAM to reserve?
4. Why can a replica with AOF enabled still lose data?

Summary

Key takeaways

• 7.4 defaults: RDB on, AOF off; a crash loses every write after the last snapshot.
• AOF everysec loses about 1 s when the machine crashes, nearly 0 when only the process dies; always trades throughput for durability.
• Redis 7: the AOF is base + incr + manifest; with AOF present, a restart loads only the AOF.
• Replication does not replace persistence; data that must not be lost needs its source of truth in a database.

References

• Redis persistence — Redis Docs: appendfsync, multi-part AOF, switching from RDB to AOF.
• Redis replication — Redis Docs: a master without persistence wipes replica data.
• Diagnosing latency issues — Redis Docs: fork latency, THP, fsync.
• redis.conf Redis 7.4: persistence defaults.