Cold Start Behaviour of Cloudflare Workers for AI Calls

Every serverless platform’s cold start is the time to build an execution environment. On Cloudflare Workers that environment is a V8 isolate rather than a container or a microVM, which changes the number by two orders of magnitude and changes what you should worry about instead.

Isolates, not containers

A Worker does not get a filesystem, a process or a kernel namespace. It gets a V8 isolate — the same unit of separation a browser uses between tabs — inside a runtime process that is already running and already hosting many other isolates. Starting one means allocating a heap and evaluating the script, not booting anything.

Cloudflare has published the figure repeatedly: an isolate starts in under 5 milliseconds. Cloudflare’s original post on eliminating cold starts is the source. Compare that with a platform that must allocate a sandbox, present a container filesystem and start a language runtime before your first line executes, and the difference is not an optimisation, it is a different architecture with a different set of costs.

The costs it trades for are real and should be stated. There is no filesystem, so nothing loads model weights locally. The API surface is web-standard — fetch, Request, Response, ReadableStream — and Node built-ins are available only through the nodejs_compat compatibility flag, which covers a subset. Any SDK depending on native modules or on the Node stream ecosystem is not a candidate.

Hiding the start inside the handshake

Cloudflare’s trick was to move the start off the critical path entirely. When the edge receives the first packet of TLS negotiation, the ClientHello, it already knows the hostname from SNI — and can eagerly load that hostname’s Worker while the handshake completes. Since starting the isolate took about 5 ms and the handshake took longer than that, by the time the first HTTP request arrived the Worker was warm. The observable cold start was zero, not because nothing happened, but because it happened in parallel with work the connection had to do anyway.

Why that stopped being enough

Two things moved in opposite directions. TLS handshakes got faster — TLS 1.3 needs one round trip where TLS 1.2 needed three — and Worker scripts got larger, so the window shrank while the thing being hidden inside it grew.

Cloudflare’s follow-up post, published on 26 September 2025, describes what they did about it and gives the numbers. That post reports that script size limits had risen to 10 MB compressed for paying users and 3 MB for free users, and that the startup CPU time limit had gone from 200 ms to 400 ms. The fix was sharding: a consistent hash ring over the machines in a data centre, so a request for a Worker is forwarded to a machine that already has that Worker loaded instead of starting a new instance locally.

The reported effects are worth reading as a description of what a cold start now is on the platform. Cloudflare states that around 4% of enterprise traffic required sharding, that the global eviction rate fell by a factor of ten, that the enterprise warm request rate went from 99.9% to 99.99% — cold starts from 0.1% to 0.01% of requests — and that the added latency from proxying to another machine is less than a millisecond.

The limit that actually binds an AI worker

Given a cold start measured in single-digit milliseconds and hidden behind a handshake most of the time, the cold start is not your problem. The startup CPU time limit is.

Everything at the top level of a Worker script runs during isolate startup and is charged against that budget: module evaluation, every import, every constant built at load, every client instantiated at module scope. A provider SDK that constructs validation schemas or builds large lookup tables when it is imported consumes the budget before your code has run. Exceeding it is a deploy-time or startup-time failure with a message about exceeding startup CPU time, not a slow request — which is a good failure mode, because it happens to you rather than to a user, but it does mean the fix has to happen before release.

The mitigations are structural rather than configurable. Import dynamically inside the handler what is only needed on some paths, so the cost lands in request CPU rather than startup CPU. Prefer building the provider request with fetch and a plain object over pulling in a full SDK for one endpoint. Keep module-scope work to constants that are genuinely constant. And remember the script size limit is on the bundled, compressed script, so a bundler that tree-shakes properly is doing platform-limit work, not just download-size work.

What this means for a model call

Put the numbers next to each other. A Worker’s cold start is under 5 ms, usually zero as observed. Time to first token from a hosted model is measured in hundreds of milliseconds to seconds. The platform’s startup cost is not a rounding error in that budget — it is beneath the noise floor of the thing you are calling.

That has one clean design consequence. On a container platform, the pressure to keep instances warm pushes you towards fewer, longer-lived, fatter services. On Workers that pressure does not exist, so splitting an AI proxy into small Workers costs nothing in startup and buys independent deployment and independent limits. The constraints that replace it are the CPU time budget per request, the subrequest limit, and the fact that a long model call must be streamed rather than buffered — a Worker that awaits a full completion before responding is holding a connection open for the whole generation.