Hard · 14 steps · ~60 min

Design an agent platform

Customers define agents and run them on your platform. Isolate tenants, meter and bill accurately, sandbox their tools and cap runaway spend while models and machines fail.

Your company sells an agent platform. Customers (tenants) define agents, with instructions, tools and knowledge, then run them through an API or on triggers such as a schedule or an incoming event. Earlier labs designed one agent for one company (support-agent), a team of cooperating agents (multi-agent-workflow) and a checking pipeline (moderation-pipeline). Here you design the layer underneath them all: many tenants share the fleets, run code you did not write, and are billed for what they use. One noisy tenant, one leaked credential or one wrong invoice is your problem, not theirs.

Same problem as Agent-building platformsWorkflow automation with AI

How it works

• A run is a loop, so it lives on a queue and an agent-runtime worker fleet; the API only accepts it. Load is counted in internal requests (model calls, tool calls, state writes), not in customer calls.
• Tenants share fleets, so every limit, queue and counter is per tenant, and big tenants can move to dedicated fleets. One tenant must never be able to slow or bill another.
• Tools are customer-defined and therefore untrusted. They run in a sandboxed tool-runner fleet with no ambient credentials and a controlled way out to the network.
• Every billable action becomes an immutable usage event in an event log, aggregated into a time-series store and reconciled, so the invoice can be defended.
• Models and machines fail: a hosted provider, a tool runner, the primary of the run-state store. Capacity, fallbacks and spares are sized for the loss.

What you'll build

1. The platform API and its data model
2. Queue, agent-runtime workers, LLM gateway and hosted models
3. A sandboxed tool-runner fleet and a run-state store
4. Usage metering into a time-series store, and an approval queue
5. A limiter and breakers against a noisy tenant flood
6. A design that survives a provider outage, a lost tool runner and a lost state-store primary

Steps

1. 1Design the platform API
2. 2Design the data model
3. 3Run the agent loop off the request path
4. 4Noisy neighbours and tenant isolation
5. 5Add a sandboxed tool fleet and a run-state store
6. 6Sandbox isolation and egress control for customer tools
7. 7Meter usage into a time-series store and add approvals
8. 8Metering and billing accuracy
9. 9Survive a noisy tenant flood
10. 10Runaway-agent protection: spend and step caps per tenant
11. 11Survive a model-provider outage
12. 12Secrets and per-tenant credentials
13. 13Lose a tool runner
14. 14Peak day, and the state-store primary fails