Build one shared AI platform for dozens of teams: gateway, retrieval, evaluation, guardrails, GPUs, metering and chargeback, sized for rising load and the worst day, inside a budget.
How it works
›Sharing makes capacity cheaper (one busy fleet, one cache) and failures wider (one bad deploy hurts every team), so the platform needs quotas, priorities and isolation.
›Inside the platform one call becomes many: guardrails, cache, retrieval, model, metering. This lab counts requests per second including those internal calls.
›Money is a requirement: every call is metered to a team, and the monthly bill must fit a budget on the worst day too.
›Metering and evaluation are essential but must never block a user request, so they hang off a topic beside the request path.
What you'll build
The platform API and its data model
A shared front door: gateway, quotas, cache, hosted and GPU models
Shared retrieval and guardrails
Metering, budgets and an evaluation sampler beside the request path
Designs that survive a provider outage, a cold cache, a lost GPU and a lost metering node