Medium · 13 steps · ~75 min

Design a video platform

Upload once, stream to millions at every quality.

Viewers stream videos far more than creators upload them. Streams need to be fast everywhere; uploads need heavy processing but can happen in the background.

Same problem as YouTubeVimeoTwitch

How it works

• Watch: a nearby edge cache serves video chunks, falling back to the origin.
• Upload: the file is stored, then workers convert it to several qualities.

What you'll build

1. The API and the data behind it
2. A system that answers 300 requests a second
3. A CDN, cache and replicas for tens of thousands
4. An upload pipeline with a queue, workers and transcoders
5. Designs that survive a cold cache, a lost server and a lost primary

Steps

1. 1Design the API
2. 2Design the data model
3. 3Handle 300 requests/sec
4. 4Handle 5,000 requests/sec for $1,000 a month
5. 5Add an upload pipeline
6. 6Handle 40,000 requests/sec
7. 7Keep p99 latency under 150 ms
8. 8Handle 150,000 requests/sec
9. 9Survive a CDN and cache restart
10. 10Lose an API server
11. 11Lose the primary database
12. 12A video that never finishes processing
13. 13Deep dive: why several qualities?