An Azure Service Bus Queue for Buffering Model Requests

Service Bus holds a message under an exclusive lock while you process it. Microsoft documents the default lock duration as one minute and the maximum as five. A model call that can take longer than five minutes therefore cannot be handled by setting a bigger number, and that single fact shapes the whole worker.

Intake, queue, worker

An HTTP-triggered function accepts the request, writes a pending row, and sends a message with the job id. A Service Bus queue holds it. A worker — a Service Bus-triggered function, a Container Apps job, or a long-running ServiceBusProcessor host — receives in PeekLock mode, calls the model, writes the result, and completes the message.

The reason to reach for Service Bus rather than Storage queues here is the settlement model. Microsoft documents two receive modes: ReceiveAndDelete, where the message is considered settled the moment the broker puts it on the wire, and PeekLock, where the receiver settles explicitly. ReceiveAndDelete on a model workload means a worker crash loses a request the user has already been told is queued. Use PeekLock.

The lock, and the five-minute ceiling

Microsoft documents the default lock duration as one minute, settable at the queue or subscription level, with a maximum value of five minutes. When the lock is explicitly released or expires, the message goes back to the front of the retrieval order for redelivery — not to the back, which means a poisonous slow message returns immediately rather than politely waiting its turn.

The lock is also documented as volatile. It can be lost during a service update, an OS update, if you change properties on the entity while holding the lock, or if the client loses its connection for any reason. When that happens the SDK raises MessageLockLostException, and notably the delivery count is not incremented — so a repeatedly lock-losing message will not dead-letter its way out of your queue on delivery count alone.

One more thing to check before you write any code: Microsoft has announced the retirement of the older WindowsAzure.ServiceBus, Microsoft.Azure.ServiceBus and com.microsoft.azure.servicebus libraries, along with the SBMP protocol, on 30 September 2026. New work belongs on Azure.Messaging.ServiceBus, which is what the examples here use.

Renewing, automatically and by hand

Because the ceiling is five minutes, a longer call must renew the lock while it runs. The SDK gives you both a manual operation, RenewMessageLockAsync, and an automatic facility on the processor where you specify a duration for which you want the lock kept renewed. The automatic route is almost always the right one, because getting the renewal cadence right by hand while also awaiting a network call is fiddly and the failure is silent.

var client = new ServiceBusClient(fqdn, new DefaultAzureCredential());

var processor = client.CreateProcessor("inference-jobs", new ServiceBusProcessorOptions
{
    ReceiveMode = ServiceBusReceiveMode.PeekLock,
    MaxConcurrentCalls = 4,
    AutoCompleteMessages = false,
    MaxAutoLockRenewalDuration = TimeSpan.FromMinutes(20),
});

processor.ProcessMessageAsync += async args =>
{
    var job = args.Message.Body.ToObjectFromJson<Job>();
    if (AlreadyDone(job.JobId)) { await args.CompleteMessageAsync(args.Message); return; }

    try
    {
        await WriteResultAsync(job.JobId, await CallModelAsync(job.Prompt));
        await args.CompleteMessageAsync(args.Message);
    }
    catch (ModelPermanentError ex)
    {
        await args.DeadLetterMessageAsync(args.Message, "PermanentModelError", ex.Message);
    }
};

Two details in that block matter. AutoCompleteMessages is off, because the default completes the message when your handler returns without throwing — which will happily complete a message whose result you failed to persist. And the permanent-error branch calls DeadLetterMessageAsync with a reason, rather than letting the message retry ten times against a provider that has already told you the request is invalid.

Microsoft is explicit about one more trap: settle a message while the receiver and its connection are still open. Close the receiver before settling and the settlement never reaches the service, the lock expires and the message is redelivered. In practice that means not disposing the processor inside a shutdown path that races your in-flight handler.

Delivery count, dead-lettering and duplicates

When receivers repeatedly abandon a message or let its lock elapse a defined number of times — the queue’s max delivery count — Service Bus moves the message to the associated dead-letter queue automatically, provided the dead-letter feature is enabled on the entity. That is your backstop, and it only works if the failures increment the counter, which lock loss does not.

For duplicates, Microsoft’s documented mechanism is the message-id: the sender sets it to a unique value aligned with an identifier from the originating process, and the worker ignores a second occurrence of a job it has already done. Service Bus also offers built-in duplicate detection on an entity, which discards repeat sends of the same message-id within a configurable history window — useful, but it protects against a duplicate send, not a duplicate delivery after a lost lock. Only your own check does that, and it is the same check described in idempotency keys for a queued model request.

Building it

1. Create the namespace and queue with the Azure CLI, setting --lock-duration to PT5M, --max-delivery-count to a value you have thought about, and enabling dead-lettering.
2. Give the worker’s managed identity the Azure Service Bus Data Receiver role on the queue and the intake function the Data Sender role. Do not use a connection string with Manage rights for either.
3. In the intake function, set each message’s MessageId to your job id before sending, so duplicate detection and your own idempotency check key on the same value.
4. Build the worker on ServiceBusProcessor with AutoCompleteMessages off and MaxAutoLockRenewalDuration above your worst-case call.
5. Test the renewal by making one call sleep past five minutes. Without renewal you will see a second delivery; with it you will not. Run this once so you know which behaviour you have.
6. Check the dead-letter subqueue has messages in it after you feed the worker a deliberately invalid request, with your reason string attached.

az servicebus queue create \
  --resource-group rg-inference --namespace-name sb-inference \
  --name inference-jobs \
  --lock-duration PT5M \
  --max-delivery-count 5 \
  --enable-dead-lettering-on-message-expiration true