Module 3 / Caching / Why cache at all?

Lesson7 minModule 3 · lesson 1 of 10

Why cache at all?

The cost of repeating work and the idea of keeping recent answers close.

Picture a food-delivery app at 7pm. Thousands of people open the same restaurant page for the same popular pizza place. Every request runs the same database query: fetch the restaurant, its menu, its rating. The answer is identical for all of them, yet the database does the work thousands of times. A query that takes 50 ms against the database takes about 1 ms from memory on a nearby machine, so repeating slow work is the most common waste in a growing system.

AppCacheDatabase1. ask2. on miss
A cache sits between the app and the slow store.

A request answered from the cache is a hit; one that must go to the database is a miss. The share of requests that hit is the hit ratio. Caching works because real traffic is lopsided: a small set of popular items receives most of the requests. A cache that holds only the top few percent of items can still absorb most of the traffic.

Life of one request

Step 1 of 3

First request: a miss

The app asks the cache for restaurant:42. It is not there, so the app queries the database (50 ms), returns the answer, and stores a copy in the cache.

Question 1

What is a cache hit?

Question 2

Why can a small cache absorb a large share of traffic?

What to remember

  1. A cache keeps recent answers close to avoid repeating slow work.

  2. A hit is served from the cache; a miss goes to the slower source and then fills the cache.

  3. A higher hit ratio means less load on the database and lower latency.

  4. A cache is only a copy: it can be stale or empty, and the system must still work without it.