Hard · 13 steps · ~50 min

Design embedding-based personalization

A content app recommends items to millions of users. Size candidate generation, the reranker, caches and the embedding pipeline within a budget, and survive a cold cache and the loss of a vector node.

A content app recommends videos and articles to millions of users. Behaviour events (views, clicks, skips) stream in; workers turn them into user and item embeddings, and a GPU fleet refreshes item embeddings in batch. To serve a request, a vector index finds a few hundred candidates near the user's embedding (candidate generation), a reranker orders them using features from a feature store, and a per-user cache answers repeat visits. You must design the API and data, then size each part within a monthly budget, survive a cold cache after a deploy and the loss of a vector node, and reason about cold start, freshness, approximate-search recall and the feedback loops a recommender creates.

Same problem as Content and product recommendationsFeed ranking

How it works

• Embed: users and items become vectors so that similar taste or content lies close together.
• Generate: an approximate nearest-neighbour search returns a few hundred candidates in milliseconds.
• Rank: a heavier model orders the candidates with fresh features, then diversity rules adjust the list.
• Learn: events and exposure logs flow back through a stream into new embeddings, closing the loop.

What you'll build

1. The API, and the events, embeddings and recommendation-log data
2. The two-stage serving path with a per-user cache
3. The event stream and workers that update embeddings
4. A feature store, and a GPU fleet for batch embedding refresh
5. A cold cache and a lost vector node, then the full peak design
6. Deep dives: two stages, freshness, cold start, approximate recall, feedback loops

Steps

1. 1Design the recommendation and event API
2. 2Design the events, embeddings and recommendation log
3. 3Serve 4,000 recommendation requests a second in two stages
4. 4Deep dive: why two stages, candidates then ranking?
5. 5Update embeddings from 7,000 behaviour events a second
6. 6Deep dive: embedding freshness versus cost
7. 7Add a feature store and scale to 9,000 requests a second
8. 8Deep dive: cold start for new users and new items
9. 9Refresh item embeddings in batch on a GPU fleet
10. 10Deep dive: approximate search, recall against latency
11. 11Survive a cold cache after a deploy
12. 12Deep dive: feedback loops, filter bubbles and diversity
13. 13Put it together: 18,000 requests a second, a vector node lost