Capacity Planning & Performance Testing: A Complete Walkthrough from Estimation to Launch

Bottom Line: The Goal of Capacity Planning Is Not “Never Go Down” — It’s Knowing When You Will

Capacity planning and performance testing are the most overlooked step in “prevention” — not because people do not know they are important, but because “a quick estimate” sounds too easy, and “testing takes too long, let us ship first” is too tempting.

But a significant portion of production incidents trace back to one root cause: the system was never validated under real-world load.

This article skips theory and starts from a concrete scenario: you are launching a new project projected to have 100,000 registered users in the first month, roughly 5,000 DAU, and a peak of 2,000 concurrent users. How many servers do you need? What database configuration? How do you verify it can handle the load?

---

Step 1: Translate Business Metrics Into Technical Metrics

Before touching any server, translate business language into technical language.

Business Metric	Conversion	Technical Metric
100K registered users	≈ 10% DAU conversion	10,000 DAU
5,000 DAU	30 requests/user/day	150,000 requests/day
2,000 peak concurrent	Factor of 5	Peak QPS ≈ 350
30% AI features	50% users × 3 uses/day	AI inference peak ≈ 50 QPS

Baseline formula:

Average QPS = Daily requests / 86400 ≈ 1.7 QPS
Peak QPS = Average QPS × peak factor (5-8) ≈ 350 QPS

This is your technical target: the system must handle 350 QPS stably without degradation.

---

Step 2: Resource Estimation by Layer

Web Service Layer

A single 4C8G cloud server can typically handle 200-400 QPS for lightweight requests. But here is a common estimation trap: ignoring slow queries and third-party dependencies.

Better estimation:

Per-instance capacity = 1000ms / (avg processing time × serial dependencies)

If your API averages 20ms (including DB query + cache) with 2 downstream calls:

Capacity ≈ 1000 / (20 × 2) ≈ 25 QPS per connection

For 350 QPS, two 4C8G servers in active-standby are sufficient. Key insight: 90% of bottlenecks are not in the web server itself — they are in the downstream database and dependencies.

Database Layer

This is where capacity planning goes wrong most often. A common mistake: “My table only has 100K rows — how could a SELECT be slow?”

Consider this query:

SELECT orders.*, users.name, products.title
FROM orders
JOIN users ON orders.user_id = users.id
JOIN products ON orders.product_id = products.id
WHERE orders.status = 'pending'
ORDER BY orders.created_at DESC
LIMIT 20;

With 100K rows, a three-table JOIN, and no suitable indexes — a single query can take 500ms+. At 350 QPS with a 50-connection pool, slow queries will drain the pool quickly.

Three principles for database estimation:

1. Measure each query independently — do not assume “this query should be fast.” Run EXPLAIN.
2. Account for connection pools — 350 QPS requires evaluating pool size and per-connection processing time.
3. Plan for write amplification — one INSERT may trigger index updates, triggers, and CDC replication.

For 350 QPS, recommended starting point: 8C16G database instance (PG/MySQL), connection pool 50-80, with Redis caching.

AI Inference Layer (if applicable)

AI inference capacity planning differs fundamentally from traditional web serving — the bottleneck is GPU memory and compute, not CPU and RAM.

Estimation flow:

Memory per request = model size + KV Cache overhead
Max concurrency per GPU = (available memory - model size) / KV Cache per request × 0.65 safety

Throughput (Token/s) = GPU compute / model parameters × batch size

For a typical 14B model (FP16 ~28GB):

• Single A100 (80GB): theoretical max ≈ (80 - 28) / 2×128K ≈ 200 concurrent (theoretical, far lower in practice)
• Realistic stable concurrency: 40-60
• 500 tokens per request → ~3-5 seconds at 40 concurrency

AI capacity planning cannot be linearly scaled by adding more GPUs — it requires accounting for memory bandwidth, inter-card communication efficiency, and batch scheduling strategy. The only reliable validation is load testing with real data and real models.

---

Step 3: Load Testing — Validate Your Estimates

Estimates are done. How do you know they are correct? Load testing.

The Testing Pyramid

         /   Full-chain (end-to-end)
        /   Module-level (single service)
       /   Component-level (single endpoint/query)

Bottom-up: test each endpoint and each query independently before full-chain testing. Otherwise, one slow query can tank the entire test without telling you where to optimize.

Tool Selection

Scenario	Recommended Tool	Why
HTTP API	wrk or k6	wrk is lightweight for single-node testing; k6 supports scenarios and metrics output
Database	pgbench / sysbench	Directly simulates query load without network overhead
AI inference	vLLM benchmark_serving.py	Official tool supporting multiple request rates and concurrency modes
Full-chain	k6 + influxdb or locust	k6 is more scriptable; locust suits teams in the Python ecosystem

A Complete Load Test Workflow

# 1. Baseline: 100 QPS, 3 minutes
k6 run --vus 50 --duration 3m --rps 100 script.js

# 2. Limit test: gradually increase, find the breaking point
k6 run --vus 200 --duration 5m --rps 500 script.js

# 3. Endurance: sustained target QPS for 30 minutes
k6 run --vus 100 --duration 30m --rps 350 script.js

# 4. Burst: simulate instantaneous traffic spike
k6 run --vus 200 --duration 1m --rps 700 script.js

Four metrics you must track:

• P50 / P95 / P99 latency — flag if P99 exceeds 1000ms more than 5% of the time
• Error rate — any 5xx is a problem; check if 4xx are business logic rate limits
• CPU and memory trends — a healthy system plateaus under sustained load; continuous growth indicates a leak
• DB connections and slow queries — database state under load reveals real issues most effectively

---

Step 4: From Results to Capacity Decisions

Tests are done. How do you interpret the results?

Results Interpretation Matrix

Test Outcome	Likely Cause	Solution
QPS far below target, CPU idle	I/O bottleneck (DB, network, disk)	Check slow queries, add indexes, use connection pool, upgrade IOPS
QPS near target but P99 degrades sharply	Resource contention (pool exhaustion, lock contention)	Increase connections, optimize lock granularity, introduce read replicas
Errors start after 10 minutes of sustained load	Memory leak or GC storm	Check heap trends, find unclosed resources, tune GC
Instant 5xx under burst traffic	No rate limiting	Add server-side rate limiting (token bucket / leaky bucket), degrade non-critical features
DB CPU at 100%, queries 10× slower	Query plan degraded due to data changes	ANALYZE to update statistics, rewrite queries, add indexes or change partition key

Pre-Launch Capacity Checklist

• Target peak QPS defined and verified with 30% headroom?
• Single point of failure mitigated? At least 2-node standby?
• Database connection pool sized appropriately? 50-80 connections for 350 QPS?
• Caching strategy in place? Expected cache hit rate for hot data?
• Rate limiting and degradation implemented? Defined behavior when traffic exceeds targets?
• Scale-up plan documented? How long to add a new instance?
• AI inference: TTFT and TPOT within expected range? GPU memory utilization stable?

---

Step 5: Gradual Rollout — Validate with Real Traffic

Load testing, no matter how realistic, is still simulation. Go-live traffic is the ultimate validation.

Recommended rollout strategy:

5% → 20% → 50% → 100%

Observe 30 minutes to 2 hours at each stage before proceeding. If P99 exceeds 1000ms at any stage, pause and investigate.

Focus on three metrics on launch day:

1. Error rate — any upward trend requires immediate attention
2. P99 latency — should stabilize under 500ms
3. Database connection count — should stay within the expected range

---

Summary

Capacity planning is not about calculating “exact numbers” — it is about building a closed loop from estimation to validation:

Business metrics → Technical metrics → Resource estimation → Load testing → Capacity decisions → Gradual rollout

Run this cycle for every new system or major feature. Not every change needs full-chain testing — small changes can rely on estimation and endpoint tests. But for changes involving AI inference, database schema changes, or new external dependencies, full-chain load testing is essential.

You will not miss a launch because you spent time on load testing. You will only miss a launch because you had a production incident you did not prepare for.

---

Designing system capacity? Need a complete load testing plan? Contact us for a free consultation.

Related reading:

• Enterprise RAG Knowledge Base Guide — GPU VRAM and concurrency capacity decisions for RAG workloads
• Cloud Cost Optimization: From Bill Analysis to Architecture Governance — Capacity planning and cost optimization are two sides of the same coin
• Monitoring and Alerting Design — How to stay on top of system health post-launch
• Docker Compose in Practice — Deployment configuration and capacity management for small to medium projects
• Message Queue Comparison — Async peak shaving with message queues