Choosing a Hosting Plan for an Azure Function That Calls a Model

The usual framing is “Consumption is cheap but cold, Premium is warm but expensive”. For a function whose whole job is to wait on a model, that framing puts the emphasis in the wrong place. Cold start is a smaller share of your latency than you think, and three other limits will force the decision first.

The plans, as of now

Microsoft’s hosting comparison currently lists five options, and the shape of the list changed recently. The Consumption plan is now documented as a legacy plan; new serverless function apps are pointed at Flex Consumption. Linux on Consumption is retiring on 30 September 2028, and function apps still on the end-of-life v3 runtime on Linux Consumption stop running after 30 September 2026. Windows on Consumption is not currently affected. Microsoft, Azure Functions scale and hosting.

• Flex Consumption — serverless, scales to zero, supports virtual network integration, and offers always-ready instances to blunt cold start. Instance memory is chosen from 512 MB, 2,048 MB or 4,096 MB. Documented maximum instance count 1,000, subject to a regional subscription memory quota.
• Premium — prewarmed workers, no scale to zero, larger instances (documented at 3.5–14 GB per instance), VNet integration, custom Linux images. Billed on core seconds and memory across needed and prewarmed instances, with at least one instance per plan always kept warm.
• Dedicated (App Service plan) — you are paying for VMs; cold start stops being a concept. Requires Always On for an unbounded timeout.
• Container Apps — functions in a container, in a Container Apps environment, with GPU access available. Cold start depends entirely on whether minimum replicas is zero.
• Consumption (legacy) — 1.5 GB per instance, and the only plan with a hard execution ceiling.

Cold start, in proportion

A cold start is the platform allocating an instance, starting the Functions host, starting the language worker and loading your dependencies. Microsoft documents a separate 60-second ceiling on the language worker process starting, which is not configurable — that is the outer bound, not the typical case.

Now put it next to the work. A chat completion of a few hundred output tokens takes seconds, because generation is sequential and each token is a forward pass. Against a four-second model call, a cold start does not double your latency; it adds a tail. It moves p99, and it barely touches p50. If your alerting is on means you may not see it at all, and if your alerting is on p99 you will see it and misattribute it.

The corollary is uncomfortable for the usual advice: buying Premium purely to remove cold start is buying a permanent monthly cost to improve a percentile that your users experience as “that one was a bit slow”. Both Flex Consumption and Premium offer always ready instances, which is the cheaper version of the same fix — a small number of perpetually warm instances under a plan that still scales out on demand.

The ceilings that actually force a move

Execution time. Microsoft’s timeout table gives Consumption a default of 5 minutes and a maximum of 10. Flex Consumption, Premium, Dedicated and Container Apps all default to 30 minutes with no enforced maximum. If a single invocation orchestrates several model calls, ten minutes is reachable, and on Consumption there is no setting that raises it.

Outbound connections. This is the buried one, and it is the one most likely to hurt a model proxy. The Consumption plan is documented at 600 active and 1,200 total outbound connections per instance. Flex Consumption, Premium and Container Apps are documented as unbounded. A function holding an open HTTPS connection to a provider for the whole duration of a streaming response is not a fast in-and-out request; connections accumulate, and a per-instance cap that is generous for a database call is not generous for hundreds of concurrent long-lived streams.

Memory. Consumption is documented at 1.5 GB per instance. That is fine for a proxy and not fine for anything that tokenises large documents in process or buffers a whole response tree before returning.

Per-instance concurrency, which is a plan decision in disguise. The HTTP extension’s throttles have plan-dependent defaults, and for a wait-bound workload they are the difference between an instance serving a hundred concurrent model calls and serving as many as its sockets allow. maxConcurrentRequests defaults to 100 on Consumption and to unbounded (-1) on Premium and Dedicated; maxOutstandingRequests defaults to 200 and unbounded on the same split; and dynamicThrottlesEnabled, which rejects requests with a 429 while system counters sit above a built-in 80% threshold, defaults to true on Consumption and false on the others. You can override all three in host.json, but the defaults tell you what each plan was designed to assume, and the timeout page covers what happens when you meet them.

Two smaller documented limits occasionally decide this too. Deployment slots: Premium is documented at 3 per app, Dedicated at 1–20, Consumption at 2, and Container Apps does not support them at all — if your release process depends on a slot swap, that removes an option outright. And app density: a Flex Consumption plan hosts exactly one function app, while Premium hosts up to 100 on one plan, which changes the arithmetic entirely when you have a dozen small functions rather than one large one.

Networking is often the real answer

Microsoft’s networking table is blunt: virtual network integration for outbound traffic is supported on Flex Consumption, Premium and Dedicated, and not on the Consumption plan. Inbound private endpoints are likewise unavailable on Consumption.

If a security review says the function must reach Azure OpenAI over private networking rather than the public endpoint, the plan decision has already been made for you and cold start never entered it. Same if the vault holding your provider key is network-restricted, since the app then needs to route its outbound traffic through the virtual network to reach it at all. See VNet integration for Azure OpenAI.

Choosing

Start on Flex Consumption. It scales to zero, it integrates with a virtual network, its connection and memory limits are the ones a model proxy wants, and always-ready instances are there if the tail latency turns out to matter. Move to Premium when you need instances larger than Flex offers, a custom Linux container image, deployment slots, or more than one function app on a shared plan. Move to Dedicated when you already have underutilised App Service capacity, or when you need an App Service Environment for isolation. Reach for Container Apps when the workload wants a GPU, which the Functions plans do not offer at all.