Running AI workloads on Google Cloud
Vertex AI, Cloud Run and BigQuery ML, read from the API and the quota page rather than from the console walkthrough.
Most Google Cloud AI guidance is written as a click-path, and a click-path is the first thing to rot. The console moves, panes get renamed, whole product names change underneath a stable API — Vertex AI documentation is currently being folded into pages titled Gemini Enterprise Agent Platform while roles/aiplatform.user and gcloud ai endpoints deploy-model carry on working exactly as before. These pages are written against the surface that does not move: the CLI flags, the resource fields, the IAM permission strings and the quota metric names.
The other half of the job is the numbers. A default concurrency, an initial GPU allocation, a token-price tier boundary and a task timeout are the facts that decide whether a deployment works on the first attempt, and they are scattered across quota pages, pricing pages and release notes rather than the tutorial you are reading. Where a figure is stated here, the publisher and the date are in the sentence and the primary source is linked. Where a figure could not be sourced, the page says which console pane or which quota name to read instead of guessing.
Deploying a Model to a Vertex AI Endpoint
Upload a model, create an endpoint, deploy with a traffic split, and call it with an authenticated prediction request.
10 min read
Running Batch Prediction Jobs on Vertex AI
Point a BatchPredictionJob at BigQuery or Cloud Storage, and read the results back without assuming the rows come out in order.
9 min read
Deploying a Model From Vertex AI Model Garden
Deploy a Model Garden or Hugging Face listing to an endpoint from the CLI, and get the accelerator quota in place before you try.
9 min read
Building a Vertex AI Vector Search Index
Build an index from JSONL embeddings, deploy it to an index endpoint, and query it — with the always-on cost stated up front.
10 min read
Grounding Vertex AI Responses With Your Own Data
Attach a grounding source to a generateContent request and turn the groundingMetadata it returns into citations that line up with the text.
10 min read
Deploying a Custom Container on Vertex AI
Build a serving container that satisfies Vertex AI's health-check and prediction contract, and deploy it without a silent health-check failure.
10 min read
Fixing 429 RESOURCE_EXHAUSTED on Vertex AI
Read the quota name in the 429 message to tell a per-minute limit from a concurrency limit from shared-pool contention, then fix the right one.
10 min read
Requesting a Vertex AI Quota Increase
Find the exact quota metric, check whether it is adjustable at all, and submit a request that has the dimensions and the justification Google needs.
9 min read
Vertex AI Pricing: Reading the Per-Token Rates
The structure of the Vertex AI rate card — tiering, modality, cached and batch multipliers, and the products billed by node-hour instead of tokens.
11 min read
Service Account Permissions for Vertex AI
Build a custom role holding only aiplatform.endpoints.predict instead of granting roles/aiplatform.user, and keep the caller's identity separate from the container's.
10 min read
Deploying a GPU-Backed Container on Cloud Run
Attach an L4 to a Cloud Run service, with the region list, CPU and memory minimums and initial quota that decide whether the first deploy works.
10 min read
Cloud Run Concurrency Settings for Model Inference
Why the default of 80 concurrent requests per instance is wrong for a local forward pass and roughly right for a proxied API call.
10 min read
Setting Minimum Instances on Cloud Run to Avoid Cold Starts
Set a minimum instance floor, understand which part of the cold start it removes, and work the always-warm cost against the latency it buys.
10 min read
Streaming a Model Response From Cloud Run
Return a chunked or server-sent-event response from Cloud Run and prove it is not being buffered somewhere between your process and the browser.
10 min read
Running a Batch Model Job With Cloud Run Jobs
Run a sharded, retryable batch inference job that has no HTTP server, no port and no request timeout to fight.
10 min read
Fixing "Container Failed to Start" on Cloud Run
The deploy-time error means one of two things failed, and the message tells you which half to look at first.
9 min read
Deploying a Cloud Functions Gen2 Endpoint for a Model Call
An HTTP-triggered 2nd gen function that calls a model, and the Cloud Run-backed limits that make it behave unlike a 1st gen one.
10 min read
Fixing a Cloud Functions Timeout on a Slow Model Call
Two clocks can end a slow model call, they fail differently, and raising the wrong one changes nothing.
9 min read
Triggering a Cloud Function From a Cloud Storage Upload
An Eventarc-backed storage trigger that runs a document through a model when the object is finalized, including the IAM grant that is easy to miss.
10 min read
Creating a Remote Model in BigQuery ML for Text Generation
Registering a Cloud resource connection and a CREATE MODEL ... REMOTE object, and what that object is and is not.
10 min read
ML.GENERATE_TEXT in BigQuery: Setup and a Working Query
A generation query that runs against a real table, the STRUCT arguments that control it, and the error column that makes a green query a lie.
10 min read
BigQuery ML Pricing for Remote Model Calls
One query produces two charges on two different meters, and for text generation one of them is almost the whole bill.
9 min read
VPC Service Controls Around Vertex AI
A service perimeter that lets Vertex AI calls happen inside it while cutting the paths data could leave by.
11 min read
Private Google Access for Vertex AI Endpoints
How a VM with no external IP reaches the Vertex AI API, and the DNS override that decides which set of Google front ends it talks to.
9 min read
Storing a Provider API Key in Google Secret Manager
Mounting a secret into a Cloud Run service so a key rotation takes effect without a redeploy — which only one of the two mount types does.
9 min read
Setting a Budget Alert for Vertex AI Spend
Scoping a Cloud Billing budget to the Vertex AI service, and being honest about what a budget alert can and cannot do.
9 min read
IAM Roles for Vertex AI: What Each One Actually Grants
Reading the predefined roles by their permission lists, and finding the narrowest one that can still call an endpoint.
9 min read
Deploying an Agent With Vertex AI Reasoning Engine
Packaging an agent and deploying it to the managed reasoningEngines runtime, which is a different resource from a model endpoint.
10 min read
Deciding Whether a Cloud Run Model Service Needs Minimum Instances
The answer is a number you can compute from your own traffic, and for most model services that number says no.
10 min read
Autoscaling a Vertex AI Endpoint by Traffic
Setting replica bounds and a target utilization metric on a deployed model, and watching what the autoscaler does with them.
10 min read
Other topics
- LLM fundamentals & architecture
- Tokens, tokenization & context windows
- Prompt engineering
- Reasoning models & test-time compute
- Multimodal AI: vision, audio, video
- RAG & retrieval
- Embeddings & vector search
- AI agents & tool use
- Structured output & function calling
- Fine-tuning & post-training
- Local inference errors, string by string
- Running local models day to day
- Testing code that calls an LLM
- Snapshot and property testing for model output
- Regression suites for prompts
- Eval gates in CI
- Flaky tests against a model
- Determinism and the cost of testing
- Contract and streaming tests
- Testing tool calls and retrieval
- Inference, serving & latency
- Rolling out a prompt change
- Testing AI systems in practice
- Forecasting a time series
- Machine learning on tabular data
- Geospatial data and models
- Understanding audio that is not speech
- Understanding video
- Core computer vision tasks
- Machine learning on graphs
- Point clouds and 3D
- Evaluation, benchmarks & LLM-as-judge
- Sensor and IoT data
- Logs and event streams
- Models over biological sequences
- Machine learning on molecules
- Embedding and searching code
- Extracting invoices and purchase orders
- Receipts, statements and tax forms
- Insurance policies and contracts
- Deeds, court filings and patents
- Extracting from medical records
- Observability & LLMOps
- CVs, certificates and identity documents
- Shipping, customs and technical documents
- Meetings, email, chat and filled-in forms
- Building an extraction pipeline
- Business, property and inspection documents
- Contract clauses and insurance claims
- Regulated and compliance documents
- Consumer, travel and closing documents
- Mapping one chat API onto another
- SDK and framework migrations
- Hallucination & failure modes
- Re-embedding and model deprecation
- Cutting over between providers
- Parity gaps, shims and legacy endpoints
- Moving between model versions
- Migrating vector stores and caches
- Mapping capabilities and parameters
- Migrating pipelines and agents
- Contracts, runbooks and rollback
- Auditing a codebase before a cutover
- Compliance and fine-tune migration
- LLM cost engineering
- Routing, cost tracking and multi-tenancy
- What a migration does to your prompts
- AI security & prompt injection
- Privacy, compliance & data residency
- AI governance, policy & society
- Building reliable AI applications
- AI hardware, GPUs & compute
- Open-weight models & local inference
- AI for developers & coding agents
- AI in industry: vertical playbooks
- AGI, superintelligence, alignment & the long future
- Machine learning foundations
- NLP fundamentals & classical tasks
- Data engineering for AI
- Synthetic data & dataset curation
- AI product design & UX
- Search, ranking & recommendation
- Enterprise adoption & change management
- AI careers, skills & teams
- Reading AI research
- AI in science & discovery
- Robotics & embodied AI
- AI economics, markets & business models
- AI myths, hype & media literacy
- Context engineering
- Shipping AI features: patterns & anti-patterns
- Build it: end-to-end AI tutorials
- Python for AI: hands-on recipes
- TypeScript, React and the web
- Frameworks and SDKs
- Errors and troubleshooting
- AI facts, numbers and statistics
- The history of AI
- The maths behind AI
- Architectures beyond the transformer
- Reinforcement learning
- Diffusion and generative media
- Speech, audio and voice engineering
- Benchmarks, one at a time
- AI search visibility
- Infrastructure and operations
- Databases and storage for AI
- Knowledge graphs and structured knowledge
- Classical ML in production
- Regulation, jurisdiction by jurisdiction
- Prompt recipes and pattern library
- AI for people who do not write code
- Writing, media and creative work
- Edge and on-device AI
- Interpretability and model internals
- Field notes
- OpenAI model behaviour
- Claude model behaviour
- Gemini model behaviour
- Llama model behaviour
- Mistral model behaviour
- Qwen model behaviour
- DeepSeek model behaviour
- Cohere model behaviour
- Grok model behaviour
- Small model behaviour
- Hybrid model architectures
- Token cost by language and script
- Transliteration, romanization and script handling
- Locale-correct output
- Multilingual generation quality
- Multilingual pipelines
- The EU AI Act, article by article
- AI under the GDPR and EU data law
- US AI regulation, state and sector
- International AI governance and standards
- AI litigation and enforcement
- Running AI workloads on AWS
- Running AI workloads on Azure
- AI at the edge: Workers, Vercel and Netlify
- Serving models on Kubernetes
- Operating AI infrastructure
- Quantization formats and what they cost
- llama.cpp, flag by flag
- Ollama and the desktop local-model runtimes
- Local models on Apple Silicon
- Hardware for local inference
- Running speech and embedding models locally
- Model files, adapters and conversion
- VRAM arithmetic for local models