Long Polling and Push Delivery for a Queue Feeding a Model Pipeline

The polling-versus-push question is usually argued on latency, and for a pipeline whose unit of work takes thirty seconds the latency difference is close to irrelevant. The thing that actually differs is who decides how much work is in flight, and that decision is the one that keeps you inside a provider’s rate limit.

The two shapes, precisely

In the pull shape, your consumer calls the broker and asks for work. Amazon SQS, Google Pub/Sub pull subscriptions and RabbitMQ’s consumer model are all this. The consumer is a running process, and it takes exactly as much work as it asks for.

In the push shape, the broker calls you over HTTP. Pub/Sub push subscriptions, Cloud Tasks HTTP targets, SNS HTTPS subscriptions and EventBridge targets are this. There is no consumer process in the usual sense — there is an endpoint, and the broker decides when it is invoked.

Lambda’s SQS event source mapping is neither, or rather it is the first wearing the clothes of the second. Lambda runs pollers on your behalf that call ReceiveMessage and then invoke your function. You get a push-shaped programming model over a pull-shaped transport, which is why its concurrency controls are event-source settings rather than anything your code can influence.

Long polling and the twenty-second ceiling

Short polling is the SQS default: ReceiveMessage queries a subset of servers, chosen by a weighted random distribution, and responds immediately even when nothing is found. Amazon documents that this can return an empty response while messages exist — a false empty response — because your request only sampled some of the servers.

Long polling is in effect whenever the wait time is greater than zero, and Amazon documents the maximum long polling wait time as 20 seconds. It queries all servers rather than a subset, and returns as soon as at least one message is available. Amazon is explicit that it reduces both empty responses and false empty responses, and thereby cost.

The cost argument is worth stating in full, because it is the strongest concrete reason to change the setting. Short polling on an empty queue in a tight loop bills a request every time round. A worker polling ten times a second against an idle queue makes 864,000 billed requests a day and does no work. The same worker with a 20-second wait makes 4,320. That is a two-order-of-magnitude difference in request count for an identical workload, which is why long polling should be the default setting on essentially every queue you create.

Set it on the queue with the ReceiveMessageWaitTimeSeconds attribute so it applies to every consumer, and per call with WaitTimeSeconds where a particular consumer wants different behaviour. Amazon notes that a low non-zero value still occasionally produces empty responses, so use 20 rather than 1 unless you have a reason.

One interaction specific to slow consumers: Amazon documents that under short polling, hitting the in-flight message limit returns an OverLimit error, while under long polling it simply stops returning messages. Long polling is cheaper and quieter, and the quiet is a real diagnostic cost — a saturated pipeline looks exactly like an empty queue. That is covered in setting a visibility timeout for slow model calls.

Push, where the ack deadline becomes your timeout

Push removes the poll loop and replaces it with a hard limit you did not previously have: the broker’s delivery deadline is now the maximum duration of your work. Google documents the Pub/Sub acknowledgement deadline as defaulting to 10 seconds with a maximum of 600, and documents that you cannot modify the deadline of an individual message received through a push subscription. Cloud Tasks documents a default handler timeout of 10 minutes for HTTP targets, with a maximum of 30.

Ten minutes is generous for one call and a real ceiling for a chain of them. In the pull shape you would extend the lease and keep working; in the push shape there is nothing to extend. A pipeline that grows from one model call into three sequential ones can cross that line without anyone changing a timeout setting, and the symptom is duplicate work rather than an error.

Push also inverts the concurrency question. A pull consumer takes what it can handle. A push endpoint is called whenever the broker feels like it, bounded only by whatever your platform’s instance and concurrency settings happen to be. This is why the platform settings in Pub/Sub triggering a Cloud Run worker are not incidental configuration — they are the only backpressure in the system.

The exception is Cloud Tasks, which is push with an explicit rate limiter attached, and is the one push system where the broker throttles itself on your behalf. If you want push and you need to respect a provider quota, that is the shape to reach for.

Where the latency actually goes

Long polling returns as soon as a message arrives, so its delivery latency on a non-empty queue is not 20 seconds — it is roughly the round trip. The 20 is a ceiling on how long an empty wait lasts. The real polling latency, on a queue that goes quiet and then receives one message, is bounded by the gap between one poll returning empty and the next starting, which is small if your loop is tight.

Push delivery latency is lower, typically well under a second. On a pipeline where the work itself is 30,000 milliseconds, a few hundred milliseconds of delivery latency is a rounding error, and anybody choosing push for latency on this workload is optimising the wrong term.

Where push genuinely wins is scale-to-zero. A pull consumer must be running to poll, so it costs something while idle. A push endpoint on a scale-to-zero platform costs nothing between messages and pays a cold start on the next one — a trade that is very attractive for bursty, low-volume inference work and unattractive for steady traffic, where the cold starts are pure loss.

Which one an inference pipeline wants

• Steady traffic, a provider concurrency limit to respect, calls longer than ten minutes, or a need to extend the lease mid-call: pull, with long polling at 20 seconds and an explicit worker count.
• Bursty low-volume work where idle cost matters more than cold starts: push, with the platform’s concurrency and maximum instance settings treated as the rate limit they are.
• Push, but with a quota to respect: Cloud Tasks, where the dispatch rate and concurrency limits are queue settings rather than application code.
• Either way: at-least-once delivery is unchanged by the choice. Both shapes redeliver, and both need the same idempotency check.