Why the Model API Call, Not Lambda Init, Dominates a Cold Path

A Lambda that calls a model provider and takes four seconds gets diagnosed as a cold start roughly every time. It is usually not. The two contributions are measured separately in the logs you already have, and one of them is typically two orders of magnitude larger than the other.

Two clocks, measured separately

Every Lambda invocation writes a REPORT line to CloudWatch Logs, and on a cold invocation it carries an extra field:

REPORT RequestId: 8f4d...  Duration: 3412.55 ms  Billed Duration: 3610 ms
Memory Size: 1024 MB  Max Memory Used: 118 MB  Init Duration: 196.44 ms

Init Duration is the execution environment being created and your module-level code running. Duration is your handler. They are additive but they are not the same clock, and only the first is a cold start. In the illustrative line above, init is 196 ms of a 3.6-second request — about 5%, and the point of writing it out is that your own REPORT lines already contain the equivalent figures. Filter your logs to lines containing Init Duration and compare the two columns before optimising either.

The other number worth having is how often you pay it at all. AWS reported in April 2025, in the Compute Blog post announcing that INIT would become billed, that across production Lambda workloads “INITs (cold starts) typically occur in under 1% of invocations”.

What init actually does

The init phase downloads your code, starts the runtime, and runs everything outside the handler. AWS documents it as limited to 10 seconds; exceed that and Lambda retries the init at the time of the first invocation, under the function timeout instead.

For a function whose job is to call a model API, what lives in init is modest: an interpreter start, an SDK import, and a client construction. Two of those are worth knowing about specifically. Constructing an SDK client resolves credentials and endpoint configuration, so doing it at module level pays that cost once per environment rather than once per request — the standard advice, and correct. And importing a large SDK package is often the single biggest line in init, which is why importing a specific client rather than a whole SDK matters more than it looks.

What none of this does is scale with your workload. Init is a roughly constant cost per environment, paid once, and then amortised over every request that environment serves.

Why the model call is the long pole

The model call scales with the answer. Generation is sequential: one forward pass produces one token, so a response is a loop whose length is the number of output tokens. Total time is approximately ttft + (output_tokens / tokens_per_second), which is the same arithmetic autoregressive generation derives in general.

Put numbers to the structure without inventing any. A 400-token answer at 40 tokens per second is 10 seconds of generation. At 100 tokens per second it is 4 seconds. In neither case does a 200 ms init matter, and no plausible init figure would — AWS caps the whole phase at 10 seconds, while the generation term has no ceiling short of your function timeout. That is the mechanism: init is bounded and constant, generation is unbounded and proportional to output length.

Three consequences follow, and they are the reason this matters operationally rather than academically:

• Your timeout is set by the model, not the runtime. Lambda’s maximum function timeout is 900 seconds, and the relevant question is what happens well before it: boto3’s default read_timeout is 60 seconds, so a long generation can fail inside a function that has minutes left. Set read_timeout deliberately, and set the function timeout above it, or you will get a task-timed-out with no useful error.
• Retries multiply the long number. The SDK’s default retry mode will re-issue a request that has already spent most of your budget. On a call whose successful path is seconds, a retry is a second full generation.
• Concurrency is driven by duration. Concurrent executions equal requests per second times average duration, so a function holding open a 10-second model call needs ten times the concurrency of one holding a one-second call at the same rate. This is how a model integration quietly consumes an account’s default 1,000 concurrent executions.

What provisioned concurrency does not fix

Provisioned concurrency pre-initialises environments so the init phase is complete before a request arrives. It genuinely eliminates the init contribution — and that is the whole of what it eliminates. Since AWS caps the entire init phase at 10 seconds and places no equivalent bound on generation, the best case for provisioned concurrency on a function whose work is one model call is bounded above by whatever your own Init Duration field reports, at the cost of paying for those environments continuously whether or not anything calls them.

It is the right tool when init is genuinely large: a heavy dependency tree, a JVM, a model loaded into memory at startup, a VPC-attached function with a large package. It is close to a waste of money on a thin function that calls an HTTP API, and the way to tell which you have is the Init Duration column, not intuition. Provisioned concurrency for AI workloads covers when the arithmetic works.

Increasing memory is the same story with a twist. Lambda allocates CPU in proportion to memory, and AWS documents 1,769 MB as the point where a function gets the equivalent of one vCPU, so more memory does make init and any local work faster. It does nothing at all to the time spent waiting on a socket, which is most of a model call. AWS also documents 625 Mbps of network bandwidth per execution environment — ample for token streams, and not the constraint people assume.

Where the time actually is

If the goal is a faster user-visible response, the levers are in the generation term:

• Stream. It does not make generation faster, but the user starts reading at time-to-first-token instead of at the end. For a synchronous Lambda this means a response-streaming function URL, since a buffered response cannot stream no matter what the model does. Note that AWS documents streamed responses as capped at 200 MB and uncapped bandwidth for the first 6 MB, then 2 MBps — not a constraint for text, but real for anything else.
• Ask for fewer output tokens. This is the only lever that reduces the dominant term directly and it is usually a prompt change.
• Cache the prefix. Prompt caching reduces time-to-first-token on long shared prefixes, which is the other half of the latency and the half that grows with your system prompt.
• Stop paying for the wait. If nothing is watching, the request does not need to be synchronous at all. Moving it behind a queue, or to a batch job, removes the whole question rather than optimising it.