Routing, cost tracking and multi-tenancy
Router config, in-flight requests during a cutover, budget caps, PII handling and per-tenant provider choice.
Moving between model providers is presented as an API problem, and the API is the easy half. The hard half is everything that was implicitly one provider: a routing config whose field names were borrowed from whoever you integrated first, a spend meter that reads a usage object that only exists in one shape, an alert threshold calibrated against one reporting delay, and a fleet of open connections that were bound to a provider at the moment each request was admitted and cannot be rebound.
These pages are about that half. They assume you are building the adapter by hand and want to know what does not survive the translation — which value has no counterpart, which default differs, which field means something subtly different on the other side — and what the residual exposure is once you have done it properly. Where a number is needed, the page derives it from inputs you substitute rather than quoting one that will be wrong next quarter.
Migrating an Internal Model Router's Config Format
How to separate a homegrown router's routing decisions from its provider bindings so that adding a provider is a data change, not a code change.
10 min read
What Happens to In-Flight Requests During a Live Provider Cutover
Why flipping a routing config does not affect requests already open, and the four distinct ways those requests fail if you do not drain them.
11 min read
Draining Traffic Safely Before a Provider Cutover
A connection-draining sequence for provider cutovers: pin bindings at admission, compute a drain deadline from your own data, and cancel stragglers deliberately.
11 min read
Migrating Retry-After Header Handling Between Providers
What Retry-After actually promises, why providers express the same idea in four different header formats, and the backoff ladder that survives all of them.
10 min read
What “Context Caching” Migration Actually Requires
The structural prompt requirements each caching model imposes, why a migration can keep working while caching nothing, and how to prove a hit.
11 min read
Migrating Token Usage Alerts and Budget Caps Between Providers
How to rebuild spend alerts on your own meter rather than a provider's reporting surface, so thresholds mean the same thing after a migration.
11 min read
What a Provider's Usage Reporting Delay Means for Real-Time Budgets
Why a hard spend cap has to be enforced from the inline usage object rather than a reporting surface, with the overshoot derived from lag, poll interval and concurrency.
11 min read
Migrating an Internal Prompt Playground to a New Provider
How to add a provider to an internal prompt-testing tool by extending one adapter, and what to do about saved runs whose parameters have no counterpart.
11 min read
What Support and SLA Terms Change When You Migrate Providers
The clause types that decide what an availability commitment is actually worth, and the engineering work a claim procedure quietly requires of you.
10 min read
Migrating a Compliance Audit Trail Between AI Vendors
Which parts of an audit trail live on the vendor's side, what happens to them when the contract ends, and how to re-establish the controls without back-dating evidence.
10 min read
Mapping Request ID Tracing Between Providers for Support Tickets
Where each major API puts its per-request identifier, which SDK property exposes it, and the three situations in which the identifier you logged is not the one support needs.
9 min read
What Changes in Prompt-Level PII Handling During a Migration
Why provider-side PII handling is a platform feature rather than a model feature, what its documented carve-outs already leak, and what a compliance workflow loses when you move.
10 min read
Migrating a Redaction Pipeline That Sits in Front of an LLM Call
How to adapt a pre-call redaction step so its placeholders survive a new provider's prompt format, structured output schema and rehydration, with a validation gate that fails loudly.
11 min read
What "Zero Data Retention" Terms Mean and How They Differ
The nine dimensions a zero-data-retention clause varies along, why caching and abuse monitoring are the usual carve-outs, and what to ask about each.
10 min read
Auditing Whether You Actually Need Zero Data Retention
An argument that zero data retention is a specific control on a specific residual risk, and that adopting it as a default precaution buys the wrong protection at a real price.
10 min read
Migrating a Multi-Tenant App's Per-Tenant Provider Config
How to build a per-tenant provider configuration layer whose resolved value is a versioned object, so tenants migrate independently and a rollback is one row with a known propagation time.
11 min read
What Breaks When Two Tenants Use Different Providers
Five failures that only appear once one codebase serves tenants on two providers, starting from the symptom each one presents as.
11 min read
Other topics
- LLM fundamentals & architecture
- Tokens, tokenization & context windows
- Prompt engineering
- Reasoning models & test-time compute
- Multimodal AI: vision, audio, video
- RAG & retrieval
- Embeddings & vector search
- AI agents & tool use
- Structured output & function calling
- Fine-tuning & post-training
- Local inference errors, string by string
- Running local models day to day
- Testing code that calls an LLM
- Snapshot and property testing for model output
- Regression suites for prompts
- Eval gates in CI
- Flaky tests against a model
- Determinism and the cost of testing
- Contract and streaming tests
- Testing tool calls and retrieval
- Inference, serving & latency
- Rolling out a prompt change
- Testing AI systems in practice
- Forecasting a time series
- Machine learning on tabular data
- Geospatial data and models
- Understanding audio that is not speech
- Understanding video
- Core computer vision tasks
- Machine learning on graphs
- Point clouds and 3D
- Evaluation, benchmarks & LLM-as-judge
- Sensor and IoT data
- Logs and event streams
- Models over biological sequences
- Machine learning on molecules
- Embedding and searching code
- Extracting invoices and purchase orders
- Receipts, statements and tax forms
- Insurance policies and contracts
- Deeds, court filings and patents
- Extracting from medical records
- Observability & LLMOps
- CVs, certificates and identity documents
- Shipping, customs and technical documents
- Meetings, email, chat and filled-in forms
- Building an extraction pipeline
- Business, property and inspection documents
- Contract clauses and insurance claims
- Regulated and compliance documents
- Consumer, travel and closing documents
- Mapping one chat API onto another
- SDK and framework migrations
- Hallucination & failure modes
- Re-embedding and model deprecation
- Cutting over between providers
- Parity gaps, shims and legacy endpoints
- Moving between model versions
- Migrating vector stores and caches
- Mapping capabilities and parameters
- Migrating pipelines and agents
- Contracts, runbooks and rollback
- Auditing a codebase before a cutover
- Compliance and fine-tune migration
- LLM cost engineering
- What a migration does to your prompts
- AI security & prompt injection
- Privacy, compliance & data residency
- AI governance, policy & society
- Building reliable AI applications
- AI hardware, GPUs & compute
- Open-weight models & local inference
- AI for developers & coding agents
- AI in industry: vertical playbooks
- AGI, superintelligence, alignment & the long future
- Machine learning foundations
- NLP fundamentals & classical tasks
- Data engineering for AI
- Synthetic data & dataset curation
- AI product design & UX
- Search, ranking & recommendation
- Enterprise adoption & change management
- AI careers, skills & teams
- Reading AI research
- AI in science & discovery
- Robotics & embodied AI
- AI economics, markets & business models
- AI myths, hype & media literacy
- Context engineering
- Shipping AI features: patterns & anti-patterns
- Build it: end-to-end AI tutorials
- Python for AI: hands-on recipes
- TypeScript, React and the web
- Frameworks and SDKs
- Errors and troubleshooting
- AI facts, numbers and statistics
- The history of AI
- The maths behind AI
- Architectures beyond the transformer
- Reinforcement learning
- Diffusion and generative media
- Speech, audio and voice engineering
- Benchmarks, one at a time
- AI search visibility
- Infrastructure and operations
- Databases and storage for AI
- Knowledge graphs and structured knowledge
- Classical ML in production
- Regulation, jurisdiction by jurisdiction
- Prompt recipes and pattern library
- AI for people who do not write code
- Writing, media and creative work
- Edge and on-device AI
- Interpretability and model internals
- Field notes
- OpenAI model behaviour
- Claude model behaviour
- Gemini model behaviour
- Llama model behaviour
- Mistral model behaviour
- Qwen model behaviour
- DeepSeek model behaviour
- Cohere model behaviour
- Grok model behaviour
- Small model behaviour
- Hybrid model architectures
- Token cost by language and script
- Transliteration, romanization and script handling
- Locale-correct output
- Multilingual generation quality
- Multilingual pipelines
- The EU AI Act, article by article
- AI under the GDPR and EU data law
- US AI regulation, state and sector
- International AI governance and standards
- AI litigation and enforcement
- Running AI workloads on AWS
- Running AI workloads on Google Cloud
- Running AI workloads on Azure
- AI at the edge: Workers, Vercel and Netlify
- Serving models on Kubernetes
- Operating AI infrastructure
- Quantization formats and what they cost
- llama.cpp, flag by flag
- Ollama and the desktop local-model runtimes
- Local models on Apple Silicon
- Hardware for local inference
- Running speech and embedding models locally
- Model files, adapters and conversion
- VRAM arithmetic for local models