Labs โ€บ Design object storage (S3-like) and pastebin0 / 12

Step 1 of 12 ยท Intro

Design object storage (S3-like) and pastebin

Start with a pastebin, grow it into a durable store for files of any size.

How it works

  • โ€บWrite: create an object or upload a big file in parts (rare, large).
  • โ€บRead: fetch by key, often the same popular objects (read-heavy).
  • โ€บKeep: never lose a byte, even when disks and servers die.

What you'll build

  1. The API and the metadata-versus-blob data model
  2. A cached metadata path for a viral paste
  3. Object storage for bytes, with a CDN for reads
  4. Replication for metadata, and designs that survive a cold CDN, a lost storage node and a lost primary