Capacity Plan Generator

Eleven inputs produce one plan: peak concurrency from Little's Law, the queueing cost of your utilisation target, the rate limit that binds first, and the month growth reaches it.

Requests per day

Peak-to-average ratioBusiest minute divided by the daily mean. A consumer product with one time zone runs 3–5; a global background job is close to 1.

Input tokens per request

Output tokens per request

Mean response timeFirst byte to last, which for a streamed answer is the whole generation.

Target peak utilisationHow close to saturation you are willing to run. Queueing delay grows as 1/(1 − ρ), so 0.9 is four times worse than 0.6.

Requests-per-minute limit

RPM

Tokens-per-minute limit

TPM

Traffic growth per month

Input rate per 1MExample figure — use your own.

$ / 1M

Output rate per 1MExample figure — use your own.

$ / 1M

Concurrent requests to provision for

Little's Law puts 30.4 requests in flight at peak; running at 70% utilisation is what turns that into the number above. The token-per-minute limit binds first, and growth reaches it in about 7.7 months.

Average request rate: 2.89 req/s
Peak, at 3.0×: 8.68 req/s
In flight at peak — L = λ × W: 30.4
Provisioned at 70% utilisation: 44
Queueing delay multiplier — 1/(1 − ρ): 3.33×
Tokens per request: 1,600
RPM needed at peak: 521 of 10,000
TPM needed at peak: 833,333 of 2,000,000
Ceiling from the RPM limit: 166.67 req/s
Ceiling from the TPM limit: 20.83 req/s
Binding limit: the token-per-minute limit
Headroom above today's peak: 140%
Months of growth until it binds: 7.7
Token cost per month, at today's volume: $72,000

What this assumes: a month is 30 days. Little's Law is exact for any stable system, so the in-flight figure holds regardless of how requests are distributed — but it uses the MEAN response time, and a workload with a long tail needs more headroom than the mean suggests. The 1/(1 − ρ) multiplier is the M/M/1 result: it is the right shape rather than the right number for a multi-server queue, and it is here to show how violently delay moves as you approach saturation, not to predict a latency. Rate limits are assumed to be enforced per minute against your whole account, so any other workload on the same key comes out of these ceilings. Growth is compounded monthly on the peak.

Three numbers decide a capacity plan

The first is concurrency, and it comes from Little's Law: the number of requests in flight equals the arrival rate times how long each one takes. It holds for any stable system, with no assumptions about the distribution, which makes it the most reliable line in any plan. Ten requests a second at three and a half seconds each is thirty-five in flight — and that is the number your connection pools, your worker count and your provider concurrency have to accommodate, not the request rate everyone quotes.

The second is headroom. A system sized exactly for its load has none, and queueing delay does not degrade linearly as you approach that point: it grows like 1/(1 − ρ). At 70% utilisation the wait is about three times the unloaded service time; at 90% it is ten. This is why a system at 90% feels broken while one at 70% feels fine, why the difference is often two more workers, and why the target utilisation field is doing more work here than any other input.

The third is which rate limit binds. Providers quote both a request-per-minute and a token-per-minute ceiling, and they bind at different traffic shapes. Short requests exhaust RPM first; long ones exhaust TPM at a fraction of the request rate. Which is which is a division you can do here in a second — and it is usually not the one people assume, which is how a launch gets throttled at a request rate well under the number in the docs. Multiply the token count by growth and you get the month it happens, which is the date to open the quota conversation, not the day you hit it.

Capacity Plan Generator

Three numbers decide a capacity plan

The reasoning behind this

Related tools