Medium · 10 steps · ~60 min

Design a distributed cache

Shard it, protect the database from stampedes and survive a cold start.

A fast, shared in-memory cache sits between your application and its database. It must hold more data than one machine can, serve hundreds of thousands of reads per second, shield the database even when entries expire or nodes restart, and keep working when a node is lost.

Same problem as Redis ClusterMemcached at FacebookTwitter

How it works

• Reads check the cache first (cache-aside); on a miss the application loads from the database and refills.
• Keys are spread over cache nodes by hashing.
• Entries expire by TTL and are evicted when memory fills.

What you'll build

1. The API and entry format
2. A first cache and a sharded, layered cache tier
3. A breaker that shields the database on a cold start
4. A design that survives a lost cache node
5. The reasoning: stampedes, hot keys and consistent hashing

Steps

1. 1Design the cache API
2. 2Design the cache and its routing
3. 3Take load off the database: 8,000 requests per second
4. 4Scale to 120,000 requests per second for $2,500
5. 5Adding a node
6. 6The stampede
7. 7Survive a cache restart
8. 8Lose a cache node
9. 9One celebrity key
10. 10Eviction and writes