Fixing “Task Timed Out After N Seconds” on AWS Lambda

Task timed out after 3.00 seconds is the least ambiguous error in Lambda: the function ran for its configured timeout and Lambda stopped it. What it does not tell you is what was slow, and on a function that calls a model API the answer is usually not the function.

What the message actually says

The number in the message is your Timeout setting, not a measurement of anything. AWS documents the default as 3 seconds, adjustable in 1-second increments up to 900 seconds (15 minutes), on its configure Lambda function timeout page. Seeing 3.00 almost always means nobody ever set the value.

The execution stops where it was. Nothing after the blocking call runs, no finally block completes reliably, and whatever the function was in the middle of — a partially written S3 object, a half-updated record, an in-flight model request already being billed by the provider — stays in that state. A timeout is not a clean failure.

It is also worth being clear on what the timeout is not. It is not a limit on your model API call specifically, it is not affected by how long a downstream service takes, and raising it does not make anything faster. It is a wall-clock budget for the whole invocation.

Four timeouts, and only one is yours

On a function that calls a model API there are at least four independent clocks. Confusing them is why this error gets fixed badly.

• The function timeout. Yours, set on the function, maximum 900 seconds. Firing it produces this message in your logs.
• The SDK or HTTP client timeout. Inside your code. Firing it produces a read-timeout exception you can catch, and the invocation ends normally. This is nearly always the one you actually want to fire.
• The provider’s own timeout. Theirs. Produces an error response from them, which your code sees as an API error.
• The caller’s timeout. API Gateway’s integration timeout, an ALB idle timeout, a browser fetch. Firing this produces nothing in your Lambda logs at all — the function runs to completion, the answer goes nowhere, and you are billed for all of it. API Gateway’s integration timeout defaulted to 29 seconds and, since AWS’s June 2024 announcement, can be raised above that for Regional and private REST APIs via a quota request — which AWS notes may require reducing your account-level throttle quota.

The ordering rule follows: each layer’s timeout should be shorter than the layer outside it. If your HTTP client’s read timeout is longer than the function timeout, the client timeout can never fire, and every slow call becomes a hard kill instead of a catchable exception with a retry.

Finding out which one fired

Start with the REPORT line for the failed request. It gives you the duration and the memory used in one place:

aws logs filter-log-events \
  --log-group-name /aws/lambda/model-caller \
  --filter-pattern "Task timed out" \
  --start-time $(( ($(date +%s) - 3600) * 1000 ))

Then read what the duration is telling you. If Duration equals the configured timeout almost exactly, every time, the function was killed — something blocked indefinitely. If it varies and only sometimes exceeds the timeout, the underlying call is variable and the timeout is simply set below the tail of a real distribution. Those two have different fixes.

To find out where inside the handler the time went, instrument with the context object rather than guessing. context.get_remaining_time_in_millis() is the only reliable source of the remaining budget, and logging it either side of the model call attributes the time directly:

def handler(event, context):
    before = context.get_remaining_time_in_millis()
    resp = client.converse(modelId=MODEL_ID, messages=messages)
    after = context.get_remaining_time_in_millis()
    print(json.dumps({"model_call_ms": before - after, "remaining_ms": after}))
    return build(resp)

Better still, use the same value to bail out deliberately. A function that returns a 504 with 500 ms left is far more useful than one that is killed with no response at all — the caller gets a real answer, your error rate is accurate, and the log line explains itself:

budget_ms = context.get_remaining_time_in_millis() - 1000
if budget_ms <= 0:
    return {"statusCode": 504, "body": '{"error":"insufficient time budget"}'}

The Init-phase variant

There is a case where the message appears and your handler never ran at all. AWS documents it under Sandbox.Timedout with the same error text, and the mechanism is worth knowing because it is self-reinforcing.

When the Init phase times out, Lambda re-runs Init on the next invocation — a suppressed init. On a function with a short timeout, that suppressed init may not complete inside the timeout either, so Init times out again; or it completes but leaves too little of the budget for the handler, so Invoke times out instead. The function is stuck failing without your code being at fault. AWS’s three remedies are to raise the timeout, raise the memory (which raises CPU proportionally and so speeds up Init), or make the initialisation code cheaper.

The tell is that your first log line never appears. If the handler logs on entry and there is no such line before the timeout, you are looking at Init, and no amount of instrumenting the handler will help.

Fixing it properly

1. Set an explicit client timeout, shorter than the function timeout. Do not rely on the SDK default, which differs between SDKs and versions. In botocore the arguments are read_timeout, connect_timeout and retries, and they belong in the client construction at module scope.

   from botocore.config import Config
   
   client = boto3.client(
       "bedrock-runtime",
       config=Config(
           connect_timeout=5,
           read_timeout=45,
           retries={"max_attempts": 2, "mode": "standard"},
       ),
   )

   Note the interaction: with retries enabled, the worst case is roughly attempts × read timeout. Two attempts at 45 seconds is 90 seconds of wall clock, so the function timeout must exceed that or the retry will be killed mid-flight.
2. Set the function timeout from the tail, not the mean. Use p99 of your observed duration plus headroom. AWS’s own guidance is that a timeout close to the average duration carries a high risk of timing out unexpectedly — generation latency varies with output length, so the tail is far from the mean.
3. Cap output length. The most effective single change for a generation function. Output tokens are produced sequentially, so an unbounded max_tokens is an unbounded duration. Setting it makes the worst case computable instead of hypothetical.
4. Raise memory before raising the timeout. More memory means proportionally more CPU, which shortens Init and any CPU-bound work in the handler. See Lambda memory size and AI workload performance.
5. If the work genuinely takes minutes, change the shape. The 900-second ceiling is hard, and anything close to it behind a synchronous HTTP caller is already broken by that caller’s own timeout. Return a job ID and process asynchronously, or stream so the client sees progress — see streaming from a Lambda function URL, noting that streaming does not stop the timeout clock.