Cold Start Impact Estimator

Derives the share of requests that hit a cold start from your traffic rate and idle timeout, and the percentile at which they start to dominate.

Requests per hourAveraged over the quiet part of the day, not the peak.

Idle timeout before scale-to-zeroHow long an instance survives with no traffic. Example value — read yours off the platform.

min

Cold start penaltyExtra seconds on the first request: container start plus weights loaded. Example value — measure your own.

Warm response timeWhat the same request costs when the instance is up.

Instances traffic spreads acrossEach one idles independently, so splitting the same traffic over more of them makes cold starts more common, not less.

Share of requests that hit a cold start

8.21%

Cold starts are invisible below the 91.79th percentile and dominate everything above it.

Arrivals per minute: 0.500
Arrivals per minute, per instance: 0.500
P(gap > idle timeout) = exp(−λT): 8.21%
Requests per day: 720
Cold starts per day: 59
Mean latency added across all requests: 0.328 s
Mean response time: 0.73 s
Cold starts appear above: p91.79
Keep-warm pings per day to prevent this: 320

What this assumes: arrivals are Poisson at a constant rate, so the gap between two requests is exponential and the chance of a gap longer than the idle timeout is exp(−λT). Real traffic is not constant — it has a daily shape — so run this once with your quiet-hours rate and once with your busy-hours rate rather than with the daily average, which flatters both. Requests are assumed to spread evenly across instances; a sticky or hash-based router concentrates traffic and does better than this. Scale-up cold starts, where a burst starts a second instance while the first is warm, are not counted and are additional.

Cold starts are a percentile problem before they are a latency problem

If arrivals are random at a steady average rate, the time between them is exponentially distributed, and the chance that a gap exceeds the scale-to-zero timeout T is exp(−λT). Every such gap ends in a cold start, so that expression is also the share of requests that pay one. It is worth internalising how fast it moves: at one request a minute with a five-minute timeout, under 1% are cold; at one request every ten minutes, 61% are. The cliff is steep and it sits exactly where a low-traffic internal service lives.

The mean is the wrong statistic here. A 2% cold-start rate with a four-second penalty adds 80 milliseconds to the average, which looks like nothing on a dashboard, while the p99 is a full cold start and every user in that tail sees a service that appears broken. Read the percentile row rather than the mean: cold starts are invisible below p(1 − P) and are the entire story above it.

The instance count works the opposite way to intuition. Spreading the same traffic over four instances quarters the arrival rate each one sees, and exp(−λT/4) is much larger than exp(−λT). More replicas make a low-traffic service colder, not warmer.

The fix is either a keep-warm ping just inside the timeout, or a minimum instance count above zero — both trade idle capacity for tail latency. This tool gives you the size of what you are trading away, not the price of it; that depends on what you are billed for idleness.

Cold Start Impact Estimator

Cold starts are a percentile problem before they are a latency problem

The reasoning behind this

Related tools