Hard · 10 steps · ~75 min

Design ad click aggregation

Count billions of clicks per window, exactly once, even when events arrive late.

An ad network records every click on every ad and must show advertisers how many clicks each campaign got per minute, while also billing them accurately. Clicks arrive in huge bursts, get retried and duplicated, and sometimes show up late.

Same problem as Google AdsMeta AdsCriteo

How it works

• Ingest: accept a click quickly and acknowledge it.
• Count: group clicks into time windows per campaign.
• Query: advertisers read counts over time, mostly recent ones.

What you'll build

1. The API and the data model
2. A stream-and-workers pipeline into a time-series store
3. A dedup guard in front of the stream
4. A fleet that survives a lost worker or broker
5. Reasoning about late events, exact vs approximate counts and hot keys

Steps

1. 1Design the API
2. 2Design the data model
3. 3Count clicks at 2,000 per second
4. 4Which clock defines a window?
5. 5Scale to 50,000 clicks per second
6. 6Drop duplicate clicks
7. 7Exact or approximate?
8. 8Lose a worker at 60,000 clicks per second
9. 9Lose a time-series node at 50,000 clicks per second
10. 10Deep dive: a hot campaign