VNet Integration for Azure OpenAI and Container Apps

The goal is that a call from your container app to Azure OpenAI never traverses the public internet, and that the Azure OpenAI resource refuses anything that did. Those are two separate configurations and neither one implies the other.

Two ways to make this private

Azure OpenAI is a Microsoft.CognitiveServices account, so it inherits the Foundry Tools network model: a default action, a list of virtual network rules, a list of IP rules, and optional private endpoints. There are two supported shapes.

• Service endpoint plus network rule. The subnet gets a Microsoft.CognitiveServices service endpoint, the resource gets a rule naming that subnet, and the default action becomes Deny. Traffic still goes to the public endpoint address but travels an optimal path and carries the subnet identity. Cheapest and quickest.
• Private endpoint. The resource gets a network interface with an IP from your address space, and DNS inside the virtual network resolves the resource hostname to it. Traffic never leaves the Azure backbone. This is the stronger control and the one that also blocks data exfiltration from the virtual network.

What is not on that list is allow-listing the Container Apps environment’s outbound IPs. Microsoft documents the outbound public IP of a Container Apps environment as something that might change over time. An allow-list built on it works until it silently does not. Microsoft, Networking in an Azure Container Apps environment.

The environment’s subnet

You must supply the subnet when you create the environment; the network type cannot be changed afterwards. The subnet must be dedicated exclusively to the Container Apps environment — no other service may share it.

Documented minimum sizes differ by environment type: /27 for a workload profiles environment (the default), /23 for a Consumption-only environment (legacy). Only the workload profiles type supports user-defined routes, egress through NAT Gateway and private endpoints in the environment, so it is the one to use here. Remember that consumption GPU replicas consume one IP each, which is a real input to the sizing if this environment will also host GPU workloads.

az network vnet create \
  --resource-group rg-model --name vnet-ai \
  --address-prefix 10.20.0.0/16 \
  --subnet-name snet-aca --subnet-prefix 10.20.0.0/23

SUBNET_ID=$(az network vnet subnet show \
  --resource-group rg-model --vnet-name vnet-ai --name snet-aca \
  --query id -o tsv)

az containerapp env create \
  --name env-inference --resource-group rg-model \
  --location westeurope \
  --infrastructure-subnet-resource-id $SUBNET_ID \
  --internal-only true

--internal-only true gives the environment an internal load balancer address instead of a public one. Microsoft documents that an internal environment cannot have public network access re-enabled later, and that creating private endpoints in the environment requires public network access to be Disabled.

Service endpoint and network rules

1. Enable the service endpoint on the subnet.

   az network vnet subnet update \
     --resource-group rg-model --vnet-name vnet-ai --name snet-aca \
     --service-endpoints "Microsoft.CognitiveServices"

2. Add a network rule on the Azure OpenAI account naming that subnet.

   az cognitiveservices account network-rule add \
     --resource-group rg-model --name my-aoai \
     --subnet $SUBNET_ID

3. Flip the default action to deny. Microsoft is emphatic that network rules have no effect until you do this.

   resourceId=$(az cognitiveservices account show \
     --resource-group rg-model --name my-aoai --query id -o tsv)
   
   az resource update --ids $resourceId \
     --set properties.networkAcls="{'defaultAction':'Deny'}"

One requirement is easy to miss and produces a puzzling 403: a request that originated from the virtual network must use the custom subdomain of the resource as its endpoint. The generic regional endpoint form will not satisfy the rule. Each resource supports up to 100 virtual network rules and up to 100 IP rules, which can be combined.

Private endpoint and the DNS trap

Creating a private endpoint updates the resource’s DNS CNAME to an alias in a subdomain prefixed with privatelink, and Azure by default creates the corresponding private DNS zone linked to your virtual network. Resolved from inside the virtual network, the resource’s normal hostname now returns the private IP; resolved from outside, it still returns the public endpoint. Same connection string either way, which is the whole point.

The trap is what you put in your configuration. Microsoft warns explicitly that requests from clients to the private endpoint must specify the custom subdomain of the resource as the endpoint base URL, and must not call the internal *.privatelink.openai.azure.com URL, which exists only as part of the intermediary CNAME resolution. Reading the private DNS zone, finding a privatelink hostname and pasting it into AZURE_OPENAI_ENDPOINT is the natural mistake and it does not work.

Microsoft also notes that Azure OpenAI uses a different private DNS zone and public DNS forwarder than other Foundry Tools, so if you run your own DNS server, take the zone name from the Azure services DNS zone configuration reference rather than assuming it matches a Cognitive Services resource you configured previously.

Controlling egress, and the hostname to allow

Locking the Azure OpenAI resource down controls who may call it. The other half of the review is usually the reverse question: what may the container app call? A workload profiles environment supports user-defined routes and egress through Azure NAT Gateway, and Microsoft describes the fully locked-down shape as an internal environment in a workload profiles environment, fronted by Application Gateway for ingress and with a UDR sending all egress through Azure Firewall. Neither UDR nor NAT Gateway is available in a Consumption-only environment, which is the second reason not to use one.

NAT Gateway is also the answer to the changing-outbound-IP problem if you genuinely need a stable source address for some third party that only supports IP allow-lists. It gives the environment a fixed egress address instead of the rotating platform ones. Note that Microsoft documents changing KEDA proxy settings as unsupported, and points at workload profiles with a NAT Gateway or UDR as the supported way to send traffic through a network appliance for inspection.

One concrete hostname will catch you out during the first incident on a locked-down environment. Every container app has two URLs: the application FQDN, and a second one granting access to the log-streaming service and the console. Microsoft states that if necessary you should add https://azurecontainerapps.dev/ to the allow list of your firewall or proxy. Skip it and the app runs perfectly while az containerapp logs show and the console blade simply hang — which reads as a broken deployment at exactly the moment you can least afford to misread it.

If you are writing firewall rules rather than relying on the private path, Azure OpenAI is covered by the CognitiveServicesManagement service tag. Microsoft is specific that this tag alone only enables SDK and REST API use; reaching the Foundry portal from inside a virtual network additionally needs AzureActiveDirectory, AzureFrontDoor.Frontend, AzureResourceManager and CognitiveServicesFrontEnd. A rule set built for the API and then blamed for the portal not loading is a rule set that was never meant to load the portal.

What breaks when you turn this on

• The portal and metrics. Microsoft states that turning on firewall rules blocks incoming requests by default, and that blocked requests include those from other Azure services, from the Azure portal, and from logging and metrics services. The playground going dark is expected, not a misconfiguration.
• Other Azure services calling the resource. There is a documented exception list. Setting networkAcls.bypass to "AzureServices" lets a trusted subset through — Foundry Tools, Azure Machine Learning and Azure Search — provided their managed identities hold the right role assignment. Set it back to "None" to revoke.
• Private address ranges in IP rules. IP network rules accept public internet addresses only. Ranges starting 10., 172.16.–172.31. and 192.168. are rejected, as are /31 and /32 prefixes — configure single addresses as individual rules.
• Key Vault, if you use it. A network-restricted vault needs the same treatment, and the app needs its outbound traffic routed through the virtual network to reach it. That is a second configuration, not a consequence of this one.