Labs โ€บ order pipeline on AWS0 / 9

Step 1 of 9 ยท Intro

Design an order pipeline on AWS

Accept orders at any speed, process them at your own, and fan each one out to several systems.

How it works

  • โ€บSQS standard queues take a very high, nearly unlimited number of requests a second and store each message in several Availability Zones before acknowledging it.
  • โ€บDelivery is at least once: a message can arrive twice, and one that is not deleted becomes visible again after its visibility timeout (30 seconds by default).
  • โ€บSNS publishes each message to every subscriber, often one SQS queue each. In us-east-1 an account can publish 30,000 messages a second by default.
  • โ€บLimits come from AWS documentation (default quotas in us-east-1). Where AWS publishes no figure, such as how many requests a container or database handles, the lab states its assumption on the component.

What you'll build

  1. A direct design at 1,800 orders a second
  2. A queue and workers that absorb a 9,000 per second spike
  3. SNS fan-out to billing and email
  4. A dead-letter queue for poison messages
  5. A worker pool that survives losing a worker
  6. Idempotent processing
  7. Per-customer ordering with FIFO
  8. A database failover that the queue absorbs