Redis data structures: choose the type by use case instead of stuffing everything into strings Basic

The problem

An e-commerce shop uses Redis 7.4 for six jobs:

1. Caching user profiles.
2. Shopping carts.
3. Daily best-selling products.
4. Counting unique daily visitors.
5. Marking which users checked in on a given day.
6. A queue for light jobs such as sending email.

The easiest approach is to SET JSON into a string for everything. Then adding to a sales score means read-modify-write of the whole string, two concurrent requests overwrite each other, and counting 100,000 unique users costs several MB.

The question for this article: for each use case, which data type offers the right atomic command, acceptable complexity, and the smallest memory footprint?

What to choose? (summary first)

Use case	Structure	Main commands	Complexity	Trap
(1) User profile	Hash user:{id}	HSET, HGET, HMGET	O(1) per field	HGETALL on a large hash is O(N)
(2) Shopping cart	Hash cart:{id}, field = SKU	HINCRBY, HDEL, HEXPIRE (7.4+)	O(1) per field	No field TTL before 7.4
(3) Daily best sellers	Sorted set top:{date}	ZINCRBY, ZRANGE ... REV	O(log N), O(log N + M)	Forgetting the TTL on per-day keys
(4) Unique users/day	HyperLogLog uv:{date}	PFADD, PFCOUNT	O(1)	Estimate only, cannot list members
(5) Daily check-in	Bitmap checkin:{date}	SETBIT, GETBIT, BITCOUNT	O(1); BITCOUNT O(N)	Large, sparse IDs waste memory
(6) Light job queue	List or stream	LMOVE/LREM; XREADGROUP/XACK	O(1)	BRPOP has no ack
Simple counter	String	INCR, INCRBY	O(1)	Max value 512 MB

If you need an exact count or must list members, use a set instead of HyperLogLog.

Core concepts

Every Redis key has a type that decides which commands apply, and an encoding that is how it is stored in memory. The type is the contract with the application; Redis chooses and changes the encoding by itself.

Bitmaps and HyperLogLogs are really strings, manipulated with bit commands or PF*. A sorted set is a set where each member has a score, always kept ordered by score.

The decision tree asks “which operation do you need” first, and only then names the type.

Loading diagram…

How it works

Sorted set: a leaderboard updated in one command

ZINCRBY adds to the score and repositions the member in O(log N). No prior read is needed, so two concurrent checkouts never lose an increment (commands run sequentially, see event loop).

ZINCRBY updates the score and rank at write time; the home page reads only the top 3 members

Enlarge diagram

Checkout → sku-ao-thunZINCRBY 3

◀ Prev▶ PlayNext ▶↺ RestartStep 1/5

An order with 3 T-shirts: ZINCRBY 3 sku-ao-thun. The member does not exist yet, so it is created with score 3.

Step	Command	State of top:2026-10-24 after the step
1	ZINCRBY 3 sku-ao-thun	ao-thun 3
2	ZINCRBY 5 sku-giay	giay 5, ao-thun 3
3	ZINCRBY 4 sku-ao-thun	ao-thun 7, giay 5
4	ZINCRBY 6 sku-tui, ZINCRBY 2+1 sku-non	ao-thun 7, tui 6, giay 5, non 3
5	ZRANGE top:2026-10-24 0 2 REV WITHSCORES	Returns ao-thun 7, tui 6, giay 5

Per-day keys need an EXPIRE (for example, keep 7 days); otherwise every day leaves a sorted set behind forever.

List: a queue without ack

BRPOP takes and removes the job from the list in a single command. If the worker dies after the pop but before sending the email, the job is lost; Redis has no copy left. The LMOVE docs call this kind of queue unreliable.

The LMOVE pattern (or BLMOVE for a blocking wait):

1. The worker runs LMOVE jobs jobs:processing RIGHT LEFT: the job moves to the processing list atomically.
2. When done, LREM jobs:processing 1 <job> (the equivalent of an ack).
3. A separate process scans jobs:processing and requeues jobs stuck too long.

Step 3 is the application’s job, and a job may run twice, so side effects need idempotency.

Stream: when you need a real ack

Streams have consumer groups: XREADGROUP hands a message to one consumer and records it in the pending entries list; XACK removes it from that list; XAUTOCLAIM reassigns stuck messages to another consumer. For retries, poison messages, and long-term replay, see the Messaging section.

Operational details

Default small-encoding thresholds in Redis 7.4 (CONFIG GET '*-max-listpack-*'):

Type	Small encoding	Switches to the large encoding when
Hash	listpack	> hash-max-listpack-entries 512 fields, or a value > hash-max-listpack-value 64 bytes
Sorted set	listpack	> zset-max-listpack-entries 128, or a member > 64 bytes → skiplist
Set	intset (all integers, ≤ 512) / listpack (≤ 128, since 7.2)	Over the threshold → hashtable

A listpack is a contiguous byte array: memory-efficient, but finding a field needs a linear scan, so it is only used below the threshold.

Other limits:

• Strings and bitmaps are at most 512 MB (offset < 2^32). SETBIT with a large offset on a new key allocates all the memory before it, which can block the server.
• HyperLogLog uses at most 12 KB with a standard error of 0.81% (Redis docs); the count is an estimate.
• HEXPIRE, HTTL, HPERSIST exist since Redis 7.4.0. Redis 8.0 adds HSETEX/HGETEX to write a field with a TTL in one command.
• KEYS, and HGETALL/SMEMBERS on large keys, are O(N) and block every other client. Use SCAN, HSCAN, SSCAN in batches (see dangerous commands).

Real-world example

The shop's six use cases on one Redis

Storing 1,000 user profiles as hashes instead of 6,000 string keys cuts memory by about 3x (measurements below).

The cart is a hash with field = SKU; flash-sale vouchers expire individually with HEXPIRE. Per-day keys (top:, uv:, checkin:) all have an EXPIRE.

The demo runs on Redis 7.4.9 (examples/docker-compose.yml), uses keys demo:redis-data-structures:*, and deletes them when done:

npx --prefix examples ts-node examples/src/redis-data-structures/demo.ts

Real output, trimmed (MEMORY USAGE ... SAMPLES 0; numbers depend on version and allocator):

== 1. 1,000 user profiles: 1 hash/user vs 6 string keys/user ==
hash   : 1000 keys, encoding=listpack, total 208000 bytes (208 bytes/user)
string : 6000 keys, encoding=embstr, total 615928 bytes (616 bytes/user)
string/hash = 2.96x

== 2. Encoding changes past the threshold ==
hash 512 fields: listpack, 6224 bytes
hash 513 fields: hashtable, 32160 bytes
trimmed back to 511 fields: hashtable (does not revert to listpack)
zset 128 members: listpack
zset 129 members: skiplist

== 3. Counting unique users: HyperLogLog vs Set, 100,000 users ==
SCARD (exact): 100000
PFCOUNT (estimate): 100165, error 0.165%
MEMORY USAGE: HLL 14424 bytes (STRLEN 12304), Set 5572928 bytes

== 5. Daily check-in with a bitmap ==
BITCOUNT: 4 , STRLEN: 13 bytes
1 user id 10,000,000: BITCOUNT 1, STRLEN 1250001 bytes, MEMORY USAGE 1310808 bytes

Reading the results:

• A hash past 512 fields grows from 6,224 to 32,160 bytes, and deleting fields does not bring it back to listpack.
• HyperLogLog is off by 0.165% on this data and uses about 1/386 of the set’s memory. 0.81% is the standard error, not an upper bound.
• A bitmap containing only user 10,000,000 still takes 1.25 MB, because every byte from 0 up to that offset is allocated.

Trade-offs

Pros

• Atomic commands that match the operation (ZINCRBY, HINCRBY, PFADD), no read-modify-write needed.

• Small encodings cut memory significantly while a collection stays below the threshold.

• HyperLogLog and bitmaps count millions of elements in a few KB or a few MB.

< slot="cons">

• Large collections become big keys: O(N) commands, slow deletion and migration.

• Probabilistic structures cannot list members and are not exact.

• Lists have no ack; streams do, but the application still has to handle pending entries and duplicates.

Cons

Production checklist

1. Set an EXPIRE on every per-day key (top:, uv:, checkin:) as soon as it is created.
2. Cap collection sizes; run redis-cli --bigkeys or --memkeys periodically.
3. Block KEYS via ACL; code uses SCAN/HSCAN/SSCAN with a moderate COUNT.
4. Measure MEMORY USAGE and OBJECT ENCODING on realistic sample data before tuning listpack thresholds.
5. Use bitmaps only for dense integer IDs; for sparse IDs use a set or map to contiguous IDs.
6. A list-based queue needs a processing list and a requeue process; if you need a real ack, use a stream.
7. Confirm the Redis version before using HEXPIRE (7.4+) or HSETEX (8.0+).

Common mistakes

The most common mistake is adding to a score via read-modify-write on a JSON string:

Read-modify-write of a leaderboard stored in a string

✕ Common

const top = JSON.parse((await redis.get('top:2026-10-24')) ?? '{}');
top['sku-giay'] = (top['sku-giay'] ?? 0) + 5;
await redis.set('top:2026-10-24', JSON.stringify(top));

✓ Prefer

await redis.zincrby('top:2026-10-24', 5, 'sku-giay');
await redis.expire('top:2026-10-24', 7 * 86400, 'NX');

Why

When two checkouts both GET and then SET, one increment is lost (lost update). ZINCRBY adds within a single command.

The second common mistake involves queues:

BRPOP then process, assuming there is a retry

The job leaves Redis as soon as BRPOP returns. If the worker is killed mid-job during a deploy, the email for order 101 is gone for good. Use LMOVE with a requeue, or a stream with XACK.

Misconception	Consequence	What to do
HGETALL is always fast because it is O(1)	A hash with 1 million fields blocks the event loop	HMGET the fields you need, or HSCAN
PFCOUNT is an exact number	Reports drift and cannot be reconciled	Use a set when exactness matters
Deleting fields brings the hash back to listpack	Memory stays high	Rewrite into a new key if it needs to shrink

Self-check

1. Why do two concurrent ZINCRBY calls never lose an increment, while GET then SET does?
2. A cart hash had 520 SKUs and was trimmed to 10. What does OBJECT ENCODING return, and why?
3. Why does a bitmap containing only user ID 10,000,000 take about 1.25 MB?
4. A worker is killed right after BRPOP. Where is the job? What about with LMOVE?

Summary

Key takeaways

• Choose the type by operation: adding scores → sorted set, unique counts → HyperLogLog/set, per-ID flags → bitmap, objects → hash.
• Small encodings (listpack, intset) save memory below the threshold; past it memory jumps and never reverts by itself.
• O(N) commands on large keys block every client; use the SCAN family and cap sizes.
• Lists have no ack; use LMOVE + requeue, or a stream with a consumer group.

References

• Redis Docs — Understand Redis data types: the main commands for each type.
• Redis Docs — HEXPIRE: field TTL, since 7.4.0.
• Redis Docs — LMOVE: the reliable queue pattern.
• Redis Docs — HyperLogLog: 12 KB, 0.81% standard error.
• Redis Docs — Memory optimization: small encodings.