Lambda Memory Size and AI Workload Performance

Lambda has no CPU setting. It has a memory setting that allocates CPU in proportion, which means the field labelled “memory” is the only performance dial the platform gives you — and for a function that mostly waits on a model API, it is easy to turn it in the wrong direction for the wrong reason.

Memory is the CPU dial

AWS states the relationship directly on its configure Lambda function memory page: “Lambda allocates CPU power in proportion to the amount of memory configured.” The configurable range is 128 MB to 10,240 MB in 1-MB increments, and at 1,769 MB a function has the equivalent of one vCPU — one vCPU-second of credits per second.

Two things follow from “one vCPU at 1,769 MB” that are not usually spelled out. Below that point your function has a fraction of a core: at the 128 MB default, roughly a fourteenth of one, which is why small functions doing real work are so much slower than the same code on a laptop. And above it you have more than one core — which only helps if something in your process can use a second thread. Single-threaded Python gains nothing from the second vCPU, so the returns past 1,769 MB are much flatter than the curve below it.

Which parts of the invocation it moves

Split one invocation of a model-calling function into segments and it becomes obvious where the dial connects.

• Init — strongly CPU-bound. Importing modules, byte-compiling, constructing SDK clients, resolving credentials, establishing TLS. This is the segment memory helps most, and it is why AWS lists raising memory among its remedies for an Init-phase timeout. On a container image with a large dependency tree, it is often the largest single win available.
• Request preparation — mildly CPU-bound. Building the payload, serialising JSON, tokenising or truncating input, applying a template. Small for a short prompt; not small when you are assembling a large retrieval context from several documents.
• The model call — not CPU-bound at all. The function is blocked on a socket. Provisioning sixteen times the CPU makes a five-second generation take five seconds. This is usually the dominant segment, and it is completely unaffected.
• Response handling — CPU-bound, and sometimes surprisingly so. Parsing a large JSON response, decoding hundreds of streamed SSE frames, validating against a schema, re-serialising. On a long streamed response the per-chunk work adds up, and this segment does respond to more CPU.

The practical consequence: the more of your invocation is the third bullet, the less memory tuning does for latency. A pure proxy in front of a model API is nearly all wait, and the honest expectation is that raising memory from 512 MB to 2 GB changes its p50 very little. A function that assembles a RAG context, calls a model, and validates a structured response has real work on both ends and behaves quite differently.

Why more memory can cost less

Lambda bills roughly on GB-seconds — memory multiplied by duration. Doubling memory doubles the per-second rate. If doubling it also halves the duration, the invocation costs the same and returns twice as fast, which is a free latency improvement. If it more than halves the duration, it is cheaper.

That is precisely what happens on CPU-bound work below one vCPU, where the function is starved. It is precisely what does not happen on network-bound work: duration is unchanged, memory doubled, cost doubled, latency identical. The same change is a free win on one function and a pure waste on another, which is why the answer cannot be copied between functions and has to be measured on each one.

Measuring it on your own function

Everything you need for a first pass is already in the REPORT line of every invocation: Duration, Billed Duration, Memory Size, Max Memory Used, and Init Duration on cold starts.

aws logs filter-log-events \
  --log-group-name /aws/lambda/model-caller \
  --filter-pattern "REPORT" \
  --start-time $(( ($(date +%s) - 86400) * 1000 )) \
  --query 'events[].message' --output text

Read Max Memory Used against Memory Size first, and resist the obvious conclusion. A function using 90 MB of 1,769 MB is not over-provisioned on memory — it is buying CPU, and the memory headroom is the price of that. The only thing a low Max Memory Used tells you is that you are not going to be killed for exceeding it.

For anything you will run at volume, AWS points at AWS Lambda Power Tuning, an open-source Step Functions state machine that runs your function at several memory settings concurrently and charts cost against speed. The important property, in AWS’s own description, is that the function runs in your account performing live HTTP calls and SDK interaction — so for a model-calling function it exercises the real API, and the resulting curve reflects the real latency mix rather than a synthetic one. It also means each sweep spends real money on real inference, which is worth knowing before pointing it at an expensive model.

A shortcut when the curve is not worth measuring: for a function that is mostly waiting on an API, the sensible band is small — somewhere from a few hundred MB to about 1,769 MB, chosen so Init is not starved — and the extremes are both wrong. 128 MB starves Init badly enough to cause timeouts; 10,240 MB buys cores that a blocked socket cannot use.

Where the reasoning breaks down

• Concurrency, not memory, is the throughput lever. A function that waits on a network call holds an execution environment for the whole wait, and its throughput is bounded by concurrency rather than by CPU. Raising memory on a throughput problem raises cost and nothing else.
• Memory is per environment. At high concurrency the GB-second bill scales with concurrency as well as memory, so a generous setting chosen at low volume becomes a large number at high volume. Revisit the setting when traffic changes shape, not only when the code does.
• Ephemeral storage is separate. The /tmp allocation is configured independently between 512 MB and 10,240 MB and does not come out of the memory setting. Downloading a large document to /tmp for processing is governed by that dial, not this one.
• Compute Optimizer’s recommendations optimise for memory usage. AWS can surface memory recommendations in the console for x86_64 functions. They are a reasonable starting point and a poor final answer for a function whose memory setting is a CPU purchase — the recommendation does not know that.

The summary that survives all of it: memory buys CPU, CPU shortens Init and any real work in the handler, and neither touches the time spent waiting for a model to answer. Tune it against the segments you actually have, and if Init is the segment that hurts, compare the cost against provisioned concurrency, which removes Init from the request path entirely instead of making it faster.