Step 1 of 15 · Intro

Meet the product

Employees ask a chatbot questions that should be answered from the company's own documents: policies, runbooks, handbooks. A model alone would guess, so the system retrieves relevant passages first and gives only those to the model, which answers with citations (retrieval-augmented generation, RAG). You must design both paths: the question path (client, API, semantic cache, vector search, reranker, then the model through an LLM gateway) and the ingestion path (documents uploaded, split, embedded and indexed through a queue and workers). It has to fit a monthly budget, survive a cold cache and a model-provider outage, and respect who is allowed to read what.

RAG

Steps · 1 of 15

RAG

Labs › chatbot over company documents0 / 15 steps

Step 1 of 15 · Intro

Design a chatbot over company documents

Employees ask questions; answers come from your documents with citations. Size retrieval, reranking, cache and model within a budget, and survive a cold cache and a provider outage.

How it works

›Ingest: a document is stored, queued, split into chunks, embedded and indexed with its access list and version.
›Retrieve: the question finds candidate chunks by meaning (and keywords), filtered by what the user may read.
›Rerank: a sharper model re-scores the shortlist and keeps the best few.
›Generate: the model answers only from those passages and returns citations, or says it did not find the answer.

What you'll build

The API and the document, chunk and chat data
The question path and the ingestion path (queue and workers)
A semantic cache and a budget
Scaling to 7,000 questions a second with your own GPUs
A provider outage, a cold cache, and the full peak design
Deep dives: chunking, hybrid search, permissions, freshness, answer not found