The building blocks

Every lab is built from these 33 components. Each has a capacity, a monthly cost and a service time in the simulation. Here is what each one is, what you would use for real, which apps rely on it, and where to practise it.

Traffic

Client
Where requests come from: browsers, phones and other services.
Capacity
source
Cost
$0/mo
Service
0 ms
In the real world Browsers, Mobile apps, Other services
Used in Every product
Practise inAd click aggregation AI capstone AI code assistant API gateway
CDN
Caches copies of content in many locations close to users, so most requests never reach your servers.
Capacity
500,000/s
Cost
$200/mo
Service
5 ms
In the real world Cloudflare, Akamai, Amazon CloudFront, Fastly
Used in Netflix, Spotify, News and media sites
Practise inAPI gateway ECS Service Auto Scaling CloudFront and API Gateway throttling API Gateway, Lambda and DynamoDB
Load balancer
Spreads incoming requests across several identical servers and stops sending to the ones that fail.
Capacity
200,000/s
Cost
$50/mo
Service
1 ms
In the real world NGINX, HAProxy, AWS Elastic Load Balancing, Envoy
Used in Nearly every web service
Practise inAd click aggregation API gateway ECS Service Auto Scaling ElastiCache and Aurora Replicas
API gateway
The single front door for an API: routing, authentication, quotas and logging in one place.
Capacity
100,000/s
Cost
$120/mo
Service
5 ms
In the real world Kong, Amazon API Gateway, Apigee, Envoy
Used in Public developer APIs, Mobile app backends
Practise inAPI gateway CloudFront and API Gateway throttling API Gateway, Lambda and DynamoDB Multi-tenant SaaS
Edge compute
Runs small pieces of your code near users, so personalised or dynamic responses can also be answered at the edge.
Capacity
300,000/s
Cost
$150/mo
Service
15 ms
In the real world Cloudflare Workers, Lambda@Edge, Fastly Compute
Used in Personalised landing pages, A/B tests, Auth checks at the edge
Practise inCDN AI safety Proximity service Search engine

Compute

API server
Your application servers: they run business logic and talk to the data tier.
Capacity
10,000/s
Cost
$100/mo
Service
30 ms
In the real world Any web framework behind a process manager
Used in Every product
Practise inAd click aggregation AI capstone API gateway ECS Service Auto Scaling
Worker
Takes jobs off a queue and does slow work in the background.
Capacity
2,000/s
Cost
$80/mo
Service
100 ms
In the real world Celery, Sidekiq, AWS Batch
Used in Email sending, Image processing, Report generation
Practise inAd click aggregation AI capstone Kinesis Data Streams S3, SQS and MediaConvert
Lambda
Serverless functions: pay per use, no servers to manage, good for bursty or glue work.
Capacity
3,000/s
Cost
$20/mo
Service
80 ms
In the real world AWS Lambda, Google Cloud Functions, Cloudflare Workers
Used in Image resizing, Webhooks, Scheduled tasks
Practise inCloudFront and API Gateway throttling API Gateway, Lambda and DynamoDB AI capstone AI pipelines
Transcoder
Heavy media processing: converts uploaded video into the formats and sizes players need.
Capacity
400/s
Cost
$220/mo
Service
800 ms
In the real world FFmpeg, AWS Elemental MediaConvert
Used in YouTube, Netflix, Twitch
Practise inS3, SQS and MediaConvert Video platform
Hosted model API
A hosted language model reached through an API. Billed per token, limited by provider quotas, and the slowest and costliest step in most AI requests.
Capacity
500/s
Cost
$1500/mo
Service
1500 ms
In the real world Anthropic API, OpenAI API, Google Gemini API, Amazon Bedrock
Used in Chat assistants, Support bots, Writing tools
Practise inAI capstone AI code assistant AI capstone AI capstone
GPU inference server
Servers with GPUs that run a model you host yourself. You control cost and latency, and you carry the capacity planning.
Capacity
1,200/s
Cost
$2500/mo
Service
600 ms
In the real world NVIDIA GPUs, vLLM, NVIDIA Triton, Kubernetes with GPU nodes
Used in Self-hosted model platforms, Large inference fleets
Practise inAI capstone AI code assistant AI capstone AI capstone
Tool runner
Runs the tools an agent calls (code, searches, API calls) in an isolated sandbox with time and resource limits.
Capacity
300/s
Cost
$180/mo
Service
1200 ms
In the real world Firecracker microVMs, gVisor, E2B, Container sandboxes
Used in Coding agents, Data-analysis assistants
Practise inAI capstone AI code assistant Agent orchestration AI agents

Data

Cache
Keeps recent answers in memory so the database is not asked the same question twice.
Capacity
100,000/s
Cost
$150/mo
Service
2 ms
In the real world Redis, Memcached
Used in Facebook (Memcached), Twitter, Reddit
Practise inAd click aggregation AI code assistant API gateway ECS Service Auto Scaling
Database
The primary database: the single source of truth that accepts writes.
Capacity
6,000/s
Cost
$300/mo
Service
20 ms
In the real world PostgreSQL, MySQL, SQL Server
Used in Instagram (PostgreSQL), Shopify (MySQL)
Practise inAd click aggregation AI capstone API gateway ECS Service Auto Scaling
Read replica
A copy of the primary that serves reads and stands by if the primary fails.
Capacity
6,000/s
Cost
$250/mo
Service
20 ms
In the real world PostgreSQL streaming replicas, MySQL replicas, Amazon Aurora
Used in Read-heavy sites such as Wikipedia
Practise inAI capstone API gateway ECS Service Auto Scaling ElastiCache and Aurora Replicas
Write-behind cache
A cache that acknowledges writes immediately and flushes them to the database later, trading durability for speed.
Capacity
100,000/s
Cost
$180/mo
Service
3 ms
In the real world Redis with async persistence, Hazelcast write-behind
Used in Counters, Leaderboards, Metrics buffers
Practise inAPI gateway Delivery dispatch Distributed cache Distributed lock
Sharded store
Splits one logical database across several machines so writes scale beyond one server.
Capacity
24,000/s
Cost
$900/mo
Service
20 ms
In the real world Vitess, MongoDB sharding, Citus, CockroachDB
Used in YouTube and Slack (Vitess), Notion
Practise inAd click aggregation API gateway CloudFront and API Gateway throttling API Gateway, Lambda and DynamoDB
Semantic cache
Stores past questions and answers and returns a stored answer when a new question means the same thing, saving a model call.
Capacity
50,000/s
Cost
$200/mo
Service
15 ms
In the real world GPTCache, Redis with vector search, Gateway-level semantic caches
Used in Support bots, FAQ assistants
Practise inAI code assistant AI capstone AI capstone AI enterprise search

Specialised stores

Object storage
Object storage for large files, such as photos, videos and backups.
Capacity
50,000/s
Cost
$40/mo
Service
40 ms
In the real world Amazon S3, Google Cloud Storage, Azure Blob Storage
Used in Dropbox, Netflix, Instagram photos
Practise inKinesis Data Streams S3, SQS and MediaConvert CDN AI capstone
Search index
An index built for text queries, relevance and filters, which a plain database does badly.
Capacity
8,000/s
Cost
$220/mo
Service
25 ms
In the real world Elasticsearch, OpenSearch, Apache Solr
Used in Wikipedia, E-commerce search, Log search
Practise inKinesis Data Streams OpenSearch, ElastiCache and Aurora AI capstone Email
Time-series store
Stores timestamped measurements with heavy appends, downsampling and retention.
Capacity
40,000/s
Cost
$200/mo
Service
15 ms
In the real world Prometheus, InfluxDB, TimescaleDB
Used in Monitoring systems, IoT platforms, Financial tick data
Practise inAd click aggregation AI capstone AI capstone Log search
Graph database
Stores relationships directly so questions like friends-of-friends stay fast.
Capacity
4,000/s
Cost
$260/mo
Service
35 ms
In the real world Neo4j, Amazon Neptune
Used in Social graphs, Fraud detection, Recommendations
Vector database
Finds the nearest items in embedding space for similarity search.
Capacity
3,000/s
Cost
$300/mo
Service
40 ms
In the real world Pinecone, pgvector, Milvus
Used in Semantic search, Recommendations, AI assistants with retrieval
Practise inAI capstone AI enterprise search AI recommendations RAG
Cold storage
Very cheap, slow archive storage for data you rarely read.
Capacity
500/s
Cost
$15/mo
Service
200 ms
In the real world Amazon S3 Glacier, Azure Archive Storage
Used in Compliance archives, Old backups
Practise inLog search Message queue Metrics monitoring Object storage
Reranker
A second-stage model that re-scores retrieved passages so the most relevant ones reach the language model.
Capacity
4,000/s
Cost
$250/mo
Service
60 ms
In the real world Cohere Rerank, Cross-encoder models, Elasticsearch learned ranking
Used in Retrieval-based chatbots, Enterprise search
Practise inAI capstone AI enterprise search AI recommendations RAG

Messaging

Queue
Accepts work now so it can be done later, absorbing bursts.
Capacity
50,000/s
Cost
$60/mo
Service
3 ms
In the real world RabbitMQ, Amazon SQS, Google Cloud Tasks
Used in Email sending, Order processing, Image pipelines
Practise inAd click aggregation AI capstone AI code assistant S3, SQS and MediaConvert
Realtime gateway
Holds many open connections so the server can push updates the moment they happen.
Capacity
20,000/s
Cost
$120/mo
Service
5 ms
In the real world WebSocket servers, Socket.IO, Phoenix Channels
Used in Slack, Discord, Live scores
Practise inAI code assistant Chat app AI capstone Collaborative editor
Pub/sub topic
Publishes each message to every subscriber, so one event can drive many reactions.
Capacity
100,000/s
Cost
$100/mo
Service
3 ms
In the real world Google Cloud Pub/Sub, Amazon SNS, Redis pub/sub
Used in Notifications, Chat fan-out, Event buses
Practise inAd click aggregation SQS, SNS and workers AI capstone Message queue
Event stream
A durable, ordered log that many consumers can read and replay.
Capacity
100,000/s
Cost
$180/mo
Service
5 ms
In the real world Apache Kafka, Amazon Kinesis, Apache Pulsar
Used in LinkedIn (created Kafka), Uber, Netflix
Practise inAd click aggregation AI capstone Kinesis Data Streams Chat app

Control

Rate limiter
Caps how many requests a client may make and rejects the rest quickly.
Capacity
100,000/s
Cost
$40/mo
Service
1 ms
In the real world Envoy rate limiting, Cloudflare Rate Limiting, Redis token buckets
Used in Public APIs from Stripe, GitHub and Twitter
Practise inAd click aggregation AI capstone AI code assistant API gateway
Circuit breaker
Stops calling a dependency that is failing so it can recover and callers fail fast.
Capacity
100,000/s
Cost
$60/mo
Service
1 ms
In the real world Resilience4j, Netflix Hystrix, Envoy outlier detection
Used in Netflix, Any service with fragile dependencies
Practise inAd click aggregation AI capstone AI code assistant API gateway
Autoscaler
Adds and removes instances to follow load, so you pay for what you use.
Capacity
100,000/s
Cost
$40/mo
Service
1 ms
In the real world Kubernetes Horizontal Pod Autoscaler, AWS Auto Scaling
Used in Services with daily or event-driven peaks
Practise inAI code assistant API gateway ECS Service Auto Scaling CDN
LLM gateway
A single front door for model calls: routes requests, enforces quotas, retries, and fails over between models and providers.
Capacity
50,000/s
Cost
$150/mo
Service
8 ms
In the real world LiteLLM, Portkey, Cloudflare AI Gateway, Kong AI Gateway
Used in Companies that call several model providers
Practise inAI capstone AI code assistant AI capstone AI capstone

The building blocks

Traffic

Client

CDN

Load balancer

API gateway

Edge compute

Compute

API server

Worker

Lambda

Transcoder

Hosted model API

GPU inference server

Tool runner

Data

Cache

Database

Read replica

Write-behind cache

Sharded store

Semantic cache

Specialised stores

Object storage

Search index

Time-series store

Graph database

Vector database

Cold storage

Reranker

Messaging

Queue

Realtime gateway

Pub/sub topic

Event stream

Control

Rate limiter

Circuit breaker

Autoscaler

LLM gateway