What a Provider's Regional Endpoint Structure Means for a Migration

“Which region?” sounds like an infrastructure detail to settle after the API works. It is not: it decides your latency floor, it decides which regulator’s territory your prompts sit in, and on at least one major platform the answer is not a single region at all.

Three shapes, not one

Providers structure regional access in one of three ways, and the shape determines what your migration has to decide.

• One global hostname. A single API host, routed by anycast or by the provider’s own edge. You do not choose a region and you cannot; latency is whatever their routing gives you from wherever your compute is. Simple, and it removes the choice rather than answering it.
• Region bound to a provisioned resource. You create an account-scoped resource in a chosen region and get a hostname for it. Azure OpenAI works this way: the region is fixed when the resource is created, and the endpoint URL is a property of that resource. The same platform also makes the deployment name rather than the model name the identifier in the request path, which Microsoft calls out explicitly as a key difference from the OpenAI API (Microsoft Learn, create and deploy an Azure OpenAI resource). So a migration onto it changes both the host and the meaning of the model field.
• Region as a routing policy. You call a regional control-plane endpoint but the identifier you pass decides where inference actually happens. Amazon Bedrock’s cross-region inference profiles are this shape, and they are the one most likely to surprise a team that thought it had made a residency decision.

Where you call is not where it runs

Bedrock’s documentation defines the distinction precisely: the source Region is the one you make the request from, and the destination Region is one of several the service may route the request to. Profile identifiers carry a geography prefix — us., eu., apac. — and AWS states that a profile tied to a geography has a destination list that never changes, while a Global profile may route to any commercial region and its list grows as regions are added (AWS, supported Regions and models for inference profiles).

Two consequences that belong in a migration plan rather than a runbook. First, the same documentation notes that a request may be routed to an opt-in region you never enabled, and that inputs and outputs may be stored in those regions for abuse detection — so choosing a global profile is a data-residency decision made in a model identifier string. Second, AWS warns that a service control policy blocking any destination region in a profile causes the request to fail even when the others are permitted. A tightened SCP and a routing-policy model identifier interact, and the failure appears as intermittent errors on a subset of traffic.

Neither of these is discoverable from the request or the response. The identifier looks like a model name, behaves like a routing policy, and carries a compliance consequence, which is an unusual amount of meaning for one string to hold and an easy thing to copy from a tutorial without noticing.

The migration action is to write down, per environment, which of the three shapes you are on and what the destination set is. On the first shape the honest answer is “unknown, ask the provider”. On the third it is a list you can enumerate, and enumerating it is the only way the compliance conversation has a factual basis.

What the wrong region costs, derived

You do not need a benchmark to bound this. Light in single-mode fibre travels at roughly two-thirds of its vacuum speed, about 200,000 km per second, which is 5 microseconds per kilometre in each direction.

Assumptions: 200,000 km/s in fibre; great-circle distance;
real fibre routes run roughly 1.3-1.6x the great-circle path.

Frankfurt -> N. Virginia, great circle ~ 6,600 km
  one-way, straight line   = 6,600 x 5 us   = 33 ms
  round trip, straight line = 66 ms
  round trip, x1.4 routing  = ~92 ms

Frankfurt -> Ireland, great circle ~ 1,100 km
  round trip, x1.4 routing  = ~15 ms

Difference in RTT floor        = ~77 ms per round trip

That 77 ms is a floor imposed by physics and routing, before any queueing, before the model produces a token, and it cannot be optimised away by anything on either end. Substitute your own pair of locations; the arithmetic is the deliverable, not the numbers. Whether 77 ms matters depends entirely on the budget it lands in: it is invisible in a batch summarisation job and it is a third of the total budget in a voice assistant latency budget.

Cold connections pay it several times

The figure above is one round trip. A request on a connection that does not already exist pays several before a single byte of your prompt is sent.

Cold connection, TCP + TLS 1.3, using the ~92 ms RTT above:

  DNS (assume cached locally)          0 RTT
  TCP handshake                        1 RTT   =  92 ms
  TLS 1.3 handshake                    1 RTT   =  92 ms
  request sent, first byte returns     1 RTT   =  92 ms
                                       -----------------
  floor before any generation                  = 276 ms

Same on a warm pooled connection:      1 RTT   =  92 ms

Difference per cold connection                 = 184 ms

Which is why the region question and the pool question are the same question. A cross-region endpoint with a keepalive expiry shorter than your traffic’s inter-request gap pays the full cold cost on a large fraction of calls, and the effect is three times the size of the regional penalty itself. Sizing that is the subject of connection pool settings. TLS 1.2 costs an additional round trip over 1.3, so a client or edge that negotiates down adds another 92 ms in this example.

What to settle before the cutover

1. Identify which of the three shapes the target uses, and if it is the third, enumerate the destination set for the exact identifier you will send.
2. Compute the RTT floor between your compute and the endpoint using the arithmetic above, and compare it against the one you have today. Do this before measuring anything: if the floor alone breaks the budget, no amount of tuning helps.
3. Confirm the model or capability you need exists in that region. Regional availability lags launch, and the mismatch shows up as a model-not-found rather than as anything regional — the case in model not found errors.
4. Check whether region choice changes quota. Rate limits are usually scoped per region or per deployment, so a multi-region setup has several separate ceilings and one of them will be the binding one.
5. Record the residency position in writing: which regions may hold prompt and completion data, on what basis, and for how long.
6. Re-check identifier syntax. A geography prefix or a deployment name is part of the string you send, and it is the most common source of a cutover that fails on the first call.