Setting an SQS Visibility Timeout for Slow Model Calls

The default visibility timeout on an SQS queue is 30 seconds. A model call frequently takes longer. Everything unpleasant about running inference behind SQS follows from those two sentences being in the same paragraph.

The symptom: two workers, one message

When a consumer receives a message it stays in the queue but becomes invisible to other consumers for the visibility timeout. If you have not deleted it by the time that window closes, it becomes visible again and another consumer picks it up. Nothing errors. Nothing is logged by AWS. You get two answers written to the same job row, two entries on the provider invoice, and if the work has side effects — an email, a webhook, a row in someone’s ledger — two of those as well.

Amazon documents the default as 30 seconds, a minimum of 0 and a maximum of 12 hours. On a workload where p99 latency is a multiple of the median and the median is already tens of seconds, the default is not a conservative choice; it is a guarantee of duplication under load, because load is exactly when the provider gets slower.

Note also that this is genuinely a timeout and not a lock in the strict sense. Amazon is explicit that because SQS is at-least-once, there is no absolute guarantee a message will not be delivered more than once even within the visibility timeout period. A correctly sized timeout makes duplicates rare; only idempotency makes them free.

Picking the number

The quantity to size against is not the model call. It is the whole time from ReceiveMessage to DeleteMessage, which includes: any in-process retries your provider SDK performs on its own, the backoff between them, the time spent writing the result, and any queueing inside your own worker if you fetched a batch and are processing it serially.

A worked example. Suppose the provider’s p99 for your prompt size is 45 seconds, your SDK is configured to retry twice with a 2-second and 4-second backoff, and the result write takes under a second. Worst case is roughly three calls at 45 plus 6 seconds of backoff, so about 141 seconds. Round to 180. If you receive in batches of ten and process them one at a time, multiply by ten and you are at 1,800 — which is usually the moment to stop batching rather than the moment to set a thirty-minute timeout.

Amazon’s own advice, where you cannot bound the number, is to start shorter — two minutes is the figure in the developer guide — and implement a heartbeat that extends the timeout while processing continues. That is better than a large fixed value, because a large fixed value delays every genuinely failed message by that amount before anyone retries it.

Extending it while you work

ChangeMessageVisibility resets a single message’s timeout without touching the queue’s default. Run it on a timer alongside the model call at roughly a third of the current window, so a single missed heartbeat does not immediately release the message.

import threading, boto3

sqs = boto3.client("sqs")

def with_heartbeat(queue_url, receipt_handle, seconds, fn):
    stop = threading.Event()

    def beat():
        while not stop.wait(seconds / 3):
            sqs.change_message_visibility(
                QueueUrl=queue_url,
                ReceiptHandle=receipt_handle,
                VisibilityTimeout=seconds,
            )

    t = threading.Thread(target=beat, daemon=True)
    t.start()
    try:
        return fn()
    finally:
        stop.set()

The same API works in the other direction and that is underused. If your worker decides early that it cannot process a message — a malformed body, a provider outage detected by a circuit breaker — call ChangeMessageVisibility with a VisibilityTimeout of 0. Amazon documents this as terminating the timeout, which makes the message immediately available to another consumer rather than leaving it invisible for three minutes while your worker sits idle. On an outage that difference is the whole recovery time.

The two ceilings you will hit

Twelve hours, from first receive. Amazon documents the maximum as 12 hours and is explicit that extending the timeout does not reset that clock — it runs from when the message was first received. A heartbeat cannot keep a message invisible indefinitely. If your work genuinely needs longer, the documented answer is to break the task into smaller steps or use Step Functions, and for inference that usually means splitting a long multi-call job into one message per call.

The in-flight quota. Standard queues have a limit of approximately 120,000 in-flight messages — received but not yet deleted. Long visibility timeouts and slow consumers are precisely how you accumulate in-flight messages, so this is a real ceiling for a large fleet rather than a theoretical one. The behaviour at the limit depends on how you poll: under short polling SQS returns an OverLimit error, while under long polling it returns no error and simply stops handing out new messages until the count drops. The second is much harder to diagnose, because it looks exactly like an empty queue. Watch ApproximateNumberOfMessagesNotVisible in CloudWatch; that metric is the in-flight count.

Setting and checking it

1. Measure your provider’s p99 for your actual prompt sizes over a representative window. Not the median, and not the number on a status page.
2. Add your SDK’s retry attempts multiplied by that p99, plus the sum of its backoffs, plus the result write. That total is your floor.
3. Set the queue attribute: aws sqs set-queue-attributes --attributes VisibilityTimeout=180. If Lambda is the consumer, remember AWS validates that the function timeout does not exceed this value, and separately recommends six times the function timeout — see SQS to Lambda for asynchronous inference.
4. Add the heartbeat for anything unbounded, at a third of the window, and make sure it stops in a finally block so a crashed call does not keep extending.
5. Alarm on ApproximateNumberOfMessagesNotVisible and on the approximate age of the oldest message. The first catches the in-flight ceiling; the second catches a queue that is being redelivered in a loop.
6. Confirm the duplicate path is safe anyway. A right-sized timeout reduces duplicates; it does not eliminate them, and Amazon says so.