VPC Peering Between an Inference Service and a Vector Database

A retrieval service that queries a vector database over the public internet pays latency on the hot path of every request and puts your embeddings on a route you do not control. Peering the two VPCs fixes both, and it is a twenty-minute job — unless the CIDR blocks overlap, in which case it is impossible and you need to know that before you start.

The CIDR conversation comes first

AWS is categorical: “You cannot create a VPC peering connection between VPCs that have matching or overlapping IPv4 or IPv6 CIDR blocks”, and adds that if you have multiple IPv4 CIDR blocks the connection is refused if any of them overlap, even when you only intended to use the non-overlapping ones. That is from AWS on how VPC peering connections work.

This is not an abstract concern with a managed vector database, because the vendor provisions a VPC on your behalf and will ask you which CIDR to give it. Default VPCs and quickstart templates overwhelmingly land on 172.31.0.0/16 or 10.0.0.0/16, so the collision is close to the default outcome. Decide the block before the vendor provisions anything; changing it afterwards generally means rebuilding their side.

aws ec2 describe-vpcs \
  --query 'Vpcs[].{id:VpcId,primary:CidrBlock,all:CidrBlockAssociationSet[].CidrBlock}' \
  --output json

Run that in every account and Region involved before you ask for a peering connection, and check the whole association set rather than just the primary block — secondary CIDRs added later are the ones that get forgotten and they count.

If the overlap is unavoidable, peering is out and the alternatives are PrivateLink, which does not require route-level reachability and therefore does not care about CIDRs at all, or a transit gateway with NAT. Many managed vector databases now offer a PrivateLink service precisely because the CIDR negotiation was a recurring source of friction — check whether yours does before building this, and see PrivateLink to a model endpoint for that shape.

Creating and accepting the connection

Peering is a two-party handshake. The requester creates it; the accepter accepts it. AWS notes that a request left alone in pending-acceptance expires after 7 days — long enough that a request sent on a Friday before a vendor’s support queue takes a week is a real failure mode.

# requester side (your inference VPC)
PCX=$(aws ec2 create-vpc-peering-connection \
  --vpc-id vpc-0inference \
  --peer-vpc-id vpc-0vectordb \
  --peer-owner-id 999988887777 \
  --peer-region us-east-1 \
  --query VpcPeeringConnection.VpcPeeringConnectionId --output text)

# accepter side (the vector database account)
aws ec2 accept-vpc-peering-connection --vpc-peering-connection-id "$PCX"

With a managed vendor you usually do not run the second command — you paste the connection id, your account id and your VPC CIDR into their console or API and they accept it. Have all three ready; the CIDR is the one people go looking for mid-form.

Routes and security groups on both sides

Accepting the connection does not move any traffic. AWS states that the owner of each VPC must manually add a route pointing at the peer VPC’s address range. Both sides, every route table associated with a subnet that needs to participate.

# in the inference VPC, for each private subnet's route table
aws ec2 create-route \
  --route-table-id rtb-0inference-private \
  --destination-cidr-block 10.42.0.0/16 \
  --vpc-peering-connection-id "$PCX"

Then the security groups. The vector database’s security group must permit inbound on its port from your side. AWS notes that if both VPCs are in the same Region you can reference a security group from the peer VPC directly as a source, which is much better than a CIDR rule: it survives your subnets being renumbered and it expresses the intent. Across Regions that is not available and you fall back to the CIDR.

A detail worth checking rather than assuming: network ACLs are stateless and apply to peering traffic like any other. If the subnets involved use custom NACLs rather than the default allow-all, you need both the inbound rule and the outbound rule for the ephemeral port range, on both sides. A one-directional NACL produces a hang that looks exactly like a missing route.

DNS across a peering connection

By default, if instances on either side address each other by public DNS hostname, the name resolves to the public address — which means the traffic leaves via an internet gateway and the peering connection you just built is unused. AWS documents the fix as enabling DNS hostname resolution for the peering connection, after which the same hostname resolves to the private address.

aws ec2 modify-vpc-peering-connection-options \
  --vpc-peering-connection-id "$PCX" \
  --requester-peering-connection-options AllowDnsResolutionFromRemoteVpc=true

Two related constraints. AWS notes you cannot query the Amazon DNS server in a peer VPC, so a private hosted zone in the vendor’s account does not automatically resolve in yours — the zone has to be associated with your VPC, which is a Route 53 operation rather than a peering one. And for an inter-Region peering connection, AWS states you must enable DNS resolution support explicitly even when the CIDR falls inside the RFC 1918 private ranges. If names resolve to public addresses after everything else is right, this option is why — related symptoms are covered in DNS resolution failing for a model endpoint inside a VPC.

The limits that shape a RAG topology

Two documented rules change how a retrieval architecture has to be laid out, and both surprise people who think of peering as a network cable.

• Peering is not transitive. If your inference VPC peers with the vector database VPC and separately with a feature store VPC, those two cannot reach each other through yours. Every pair that needs to talk needs its own connection, and past three or four VPCs a transit gateway is the shape that stops the count growing quadratically.
• No edge-to-edge routing through a gateway. AWS states plainly that if VPC A has a NAT device providing internet access, resources in VPC B cannot use it; the same holds for an internet gateway, a VPN connection, Direct Connect, and an S3 gateway endpoint. For a RAG system this is the one that bites: the vector database VPC cannot reach your model provider through your NAT gateway, and cannot reach S3 through your gateway endpoint. Each VPC arranges its own egress.

Two more worth knowing before you measure anything. The MTU is 9001 bytes for same-Region peering and 8500 for inter-Region, so an inter-Region peering connection quietly fragments jumbo frames that worked in-Region. And a peering connection has no bandwidth charge of its own, but AWS bills data crossing an Availability Zone over a peering connection at $0.01/GB in each direction — so an inference service in one AZ querying a vector database in another pays per query, forever. Placing them in the same AZ is a real optimisation for a retrieval path that runs on every request.

Finally, keep the connection visible. A peering connection is invisible in every service console except VPC’s, it has no health metric, and AWS notes that even a Region event preventing traffic leaves its status at Active. Whatever tells you the vector database is unreachable will be your own application’s error rate, so make sure that error is distinguishable from a slow query rather than folded into one timeout counter.