When to move beyond one primary
The limits of a single writer.
The delivery app is now busy in many cities. Replicas handle the reads, indexes are tuned, the pool is sized and backups are drilled. Writes still all funnel into one primary. When does that stop being enough, and what do you do about it? The answer is a ladder of steps, and you should climb it from the cheapest rung.
The ladder, cheapest first
Step 1 of 5
Make each write cheaper
Add missing indexes, drop unused ones, batch many small inserts into one, and move big blobs out of the database. A primary can often do several times more after this.
How do you know it is time to climb? Use evidence, not instinct. Watch the primary's CPU, disk I/O wait, replication lag and p99 query latency at peak. If CPU sits near 80% during peaks after tuning, or the write-ahead log cannot be flushed fast enough, you are near the limit. If only reads are hot, the answer is replicas or caches, not sharding.
A back-of-envelope check helps. Suppose one primary tops out around 6,000 writes per second after tuning, and your busiest hour peaks at 4,500. You have 25% headroom, not a lot. If growth is 10% per month, you will hit the ceiling in about three months, so start the next rung now, since moving to separate databases or shards takes quarters, not days.
Your turn
Your chat history table grows by 30 GB a day and its writes are starting to slow the orders database. Which rung do you pick and why?
Stuck? Is the problem the whole database or one independent feature?
What to remember
Climb from cheap to costly: tune, scale up, cache and queue, separate, then shard.
Use measurements such as CPU, I/O wait and p99 latency to decide.
Plan the next step early because migrations take time.
Do not accept multi-writer complexity without needing it.