Retries and Error Handling in AWS Step Functions for Model Calls

Step Functions defaults to failing the entire execution when any state reports an error. For a workflow whose expensive step is a model API that throttles under load, that default turns a two-second hiccup into a lost execution and a wasted upstream charge.

What you get if you write nothing

Every state except Pass and Wait can report a runtime error, and with no Retry and no Catch the execution fails. Catchers are available on Task, Parallel and Map states, and AWS is explicit that they do not cover top-level execution failures — so a workflow that must survive its own failure needs either a caller that handles it, a parent workflow with the child nested inside a caught state, or an EventBridge rule listening for TIMED_OUT events.

Errors are identified by case-sensitive strings. Built-in ones all begin with States.; errors your code reports must not use that prefix.

The retrier fields

A Retry field is an array of retriers, scanned in order, and the first whose ErrorEquals contains the reported error name wins. The fields and their documented defaults:

• ErrorEquals — required, a non-empty array of error name strings.
• IntervalSeconds — seconds before the first retry. Default 1, maximum 99,999,999.
• MaxAttempts — maximum retry attempts. Default 3. A value of 0 means never retry this error, which is how you exclude one error from a later catch-all retrier.
• BackoffRate — the multiplier applied to the interval after each attempt. Default 2.0.
• MaxDelaySeconds — an upper bound on the interval, greater than 0 and less than 31,622,401. If you omit it, Step Functions does not cap the exponential growth at all.
• JitterStrategy — FULL or NONE. Default NONE.

Two of those defaults are worth pausing on. MaxDelaySeconds being absent by default means a retrier with a long IntervalSeconds and a high MaxAttempts can wait an absurdly long time on its last attempt — the arithmetic is exponential and nothing bounds it. And JitterStrategy defaulting to NONE means the out-of-the-box behaviour is synchronised retries, which is the opposite of what you want when a hundred executions all hit a throttle at the same instant.

Retriers accumulate per state execution rather than per error: AWS documents that a retrier’s MaxAttempts applies across all visits to that retrier within one state execution, so alternating between two error names does not reset either counter. Retries are also billed state transitions.

Which errors are worth retrying

This is where a model call differs from an ordinary Lambda invocation, because the failures split cleanly into three groups and only one of them should be retried.

Retry with backoff: throttling and capacity errors. Bedrock’s ThrottlingException and ModelNotReadyException, a provider’s HTTP 429, and the transient Lambda service exceptions AWS calls out by name — Lambda.ServiceException, Lambda.SdkClientException and Lambda.TooManyRequestsException — are all worth waiting out. These will succeed later without any change to the request.

Never retry: validation and authorisation errors. A ValidationException because the prompt exceeded the model’s context window will produce exactly the same error three more times, four seconds apart, and cost four billed state transitions to prove it. The same goes for AccessDeniedException and States.Permissions, which reports that a task lacked the privileges to run at all — no amount of waiting grants an IAM permission.

Retry carefully, if at all: timeouts. A States.Timeout on a model call means the request may well have been processed and billed upstream, and you simply stopped waiting. Retrying is not free and it is not idempotent: you pay twice and, if the operation had a side effect, it happens twice. Retry a timeout only where the call is genuinely idempotent, or where the cost of the duplicate is smaller than the cost of the failure.

Three error names carry behaviour worth memorising. States.ALL must appear alone in its ErrorEquals and must be the last retrier or catcher; it cannot catch States.Runtime or States.DataLimitExceeded. States.TaskFailed acts as a wildcard matching any known error except States.Timeout. And States.DataLimitExceeded — reported when a state’s output exceeds the payload size quota, a very real outcome when a model returns a long completion into the state — is terminal and can only be caught by naming it explicitly. That last one is the trap: a workflow with a careful States.ALL catcher will still die outright on a long response unless you name the error.

A complete state for a model call, with the three groups separated:

"InvokeModel": {
  "Type": "Task",
  "Resource": "arn:aws:states:::bedrock:invokeModel",
  "Parameters": {
    "ModelId": "anthropic.claude-3-5-sonnet-20241022-v2:0",
    "Body": {
      "anthropic_version": "bedrock-2023-05-31",
      "max_tokens": 1024,
      "messages.$": "$.messages"
    }
  },
  "TimeoutSeconds": 120,
  "Retry": [
    {
      "ErrorEquals": ["ValidationException", "AccessDeniedException"],
      "MaxAttempts": 0
    },
    {
      "ErrorEquals": [
        "ThrottlingException",
        "ModelNotReadyException",
        "ServiceQuotaExceededException"
      ],
      "IntervalSeconds": 4,
      "MaxAttempts": 6,
      "BackoffRate": 2,
      "MaxDelaySeconds": 60,
      "JitterStrategy": "FULL"
    },
    {
      "ErrorEquals": ["States.Timeout"],
      "IntervalSeconds": 10,
      "MaxAttempts": 1,
      "BackoffRate": 1
    }
  ],
  "Catch": [
    {
      "ErrorEquals": ["States.DataLimitExceeded"],
      "ResultPath": "$.errorInfo",
      "Next": "SummariseAndRetryShorter"
    },
    {
      "ErrorEquals": ["States.ALL"],
      "ResultPath": "$.errorInfo",
      "Next": "RecordFailure"
    }
  ],
  "ResultPath": "$.completion",
  "Next": "ValidateOutput"
}

The first retrier exists purely to shadow the third: because retriers are scanned in order and the first match wins, listing the non-retriable errors with MaxAttempts: 0 ahead of any broader retrier is how you exclude them. Without it, a later States.ALL would happily retry a malformed request six times.

Jitter, and why a fleet needs it

With JitterStrategy at its default of NONE, every execution that hit the same throttle at the same moment retries at the same moment, then again together, then again — a thundering herd that keeps the upstream at its limit and makes the outage longer than the event that caused it.

Setting FULL randomises each wait between zero and the computed interval. AWS’s worked example: with MaxAttempts 3, IntervalSeconds 2 and BackoffRate 2, the unjittered waits are 2, 4 and 8 seconds; with FULL, they become random draws from 0–2, 0–4 and 0–8. The mean wait halves and the concentration disappears. Pairing it with MaxDelaySeconds, as in the state above, bounds the tail so the sixth attempt is not scheduled two minutes out.

The Catch branch

Catch runs only after retries are exhausted or when no retrier matched. Each catcher takes ErrorEquals, a required Next, and an optional ResultPath. The error output delivered to the next state is an object containing a human-readable Cause field, and where Cause holds escaped JSON — which it does for many service integrations — a Pass state with States.StringToJson($.Cause) turns it back into something you can branch on.

Always set ResultPath on a catcher. In a JSONPath workflow it defaults to $, which overwrites the entire state input with the error output — so the failure branch loses the request it was handling and cannot record what failed, retry it differently, or return it to a queue. Writing to $.errorInfo keeps both.

The two-catcher shape above is the useful pattern for model work: one named branch for a failure you can do something intelligent about — here, an output too large for the payload quota, which you handle by summarising and retrying with a smaller max_tokens — and a catch-all that records and stops. A workflow where every error goes to one branch called “fail” is a workflow that has thrown away the only information it had.