Running a Batch Model Job With Cloud Run Jobs

A Cloud Run job is the same container runtime with the request-serving parts removed. No port, no health check, no 60-minute request ceiling — it runs to completion and its exit code is the result.

What a job is that a service is not

The differences are worth stating plainly because they determine which one a batch workload belongs on.

• No HTTP server. A job container does not listen on a port and is never probed. The frequent Cloud Run failure of a container that fails to start because nothing bound the port simply does not exist here.
• Exit code is the contract. Zero is success, non-zero is failure. There is no status code and no response body.
• Runs far longer. A service request is capped at 60 minutes; a job task is capped at 168 hours by Google’s documented limit. Anything that outgrows a request belongs here.
• Started, not called. A job does nothing until executed, from the CLI, from Cloud Scheduler, from Workflows or from an Eventarc trigger. There is no URL to secure and nothing to receive unwanted traffic.

Against Vertex AI batch prediction the choice is about who owns the loop. A BatchPredictionJob handles sharding, retries and output writing for you but constrains the shape of the work to a model and an input table. A Cloud Run job is arbitrary code, which is what you want when the work is “fetch, preprocess, call a model, post-process, write to three places” rather than pure scoring.

Sharding with the task index

A job runs as some number of tasks, and every task runs the same image with the same arguments. Google documents two environment variables that make them behave differently: CLOUD_RUN_TASK_INDEX, a value between 0 and the number of tasks minus one, and CLOUD_RUN_TASK_COUNT, the number of tasks. Two more are available in the execution environment, CLOUD_RUN_EXECUTION naming the execution and CLOUD_RUN_TASK_ATTEMPT counting retries of that task.

import os

index = int(os.environ["CLOUD_RUN_TASK_INDEX"])
count = int(os.environ["CLOUD_RUN_TASK_COUNT"])
attempt = int(os.environ.get("CLOUD_RUN_TASK_ATTEMPT", "0"))

rows = fetch_pending_rows()                  # deterministic ordering required
mine = rows[index::count]                    # strided, so shards stay balanced

for row in mine:
    if already_written(row.id):              # cheap idempotency check
        continue
    result = call_model(row.text)
    write_result(row.id, result, attempt=attempt)

The strided slice is deliberate. Splitting into contiguous blocks gives every shard a run of adjacent rows, and adjacent rows are usually similar — the same customer, the same day, the same document length — so one shard ends up with all the slow work and the job finishes when that one finishes. Striding distributes size variation evenly across tasks at no cost.

Creating and running it

gcloud run jobs create score-tickets \
  --image=us-central1-docker.pkg.dev/PROJECT/batch/scorer:2.0 \
  --region=us-central1 \
  --tasks=50 \
  --parallelism=10 \
  --task-timeout=2h \
  --max-retries=3 \
  --cpu=2 \
  --memory=4Gi \
  [email protected]

gcloud run jobs execute score-tickets --region=us-central1 --wait

--tasks is how many shards exist; --parallelism is how many may run at once. Keeping parallelism below the task count is the throttle that stops fifty containers hitting a provider rate limit simultaneously and converting a batch job into a wall of 429s. It is the same reasoning as bounding client concurrency against a per-model quota, applied at the fleet level instead of inside one process.

--wait makes the execute command block and return the execution’s exit status, which is what you want from CI or a scheduler that needs to know whether the run succeeded. Without it the command returns as soon as the execution is accepted.

Two more flags matter for scheduled work. Job configuration is versioned the way a service is, so gcloud run jobs update changes the template for future executions and leaves a run already in flight alone — which means a fix deployed mid-run does not apply to that run. And --args or --set-env-vars can be overridden per execution with the same flags on jobs execute, which is how one job definition serves a nightly full pass and an ad-hoc backfill over a date range without a second job existing to drift out of sync with the first.

Retries make idempotency mandatory

Google documents --max-retries as accepting 0 to 10, defaulting to 3. The important detail is what gets retried: the whole task, from the start, not the row that failed. A task that has processed 900 of 1,000 rows and then dies re-runs all 1,000 on its next attempt.

For model inference that is money. A thousand calls repeated three times is three thousand calls billed, and unlike a failed HTTP request these all succeed individually — nothing about them looks wrong on a bill. Thealready_written check in the example above is not defensive style, it is the thing that makes retries affordable, and it needs to be cheap enough to run per row and durable enough to survive the crash.

Two follow-on rules come from the same fact. Write results incrementally rather than accumulating them in memory and writing at the end, because an accumulate-then-write task loses everything on failure and has nothing to resume from. And make the write itself idempotent — an upsert keyed on the row identifier, not an append — so that a race between a dying attempt and its replacement cannot duplicate a row.

An execution is considered failed when any task exhausts its retries. Setting --max-retries=0 is the right choice when a failure means the input is bad rather than the infrastructure was unlucky, because three attempts at a malformed shard is three times the cost for the same answer.

The documented ceilings

From Google’s Cloud Run quotas and limits documentation, read on 11 August 2026: a maximum of 10,000 tasks per job; a task timeout defaulting to 10 minutes with a maximum of 168 hours, reduced to 1 hour when GPUs are attached; a maximum of 10 retries per task; and completed executions retained until there are 1,000 per job, after which the oldest are deleted automatically.

The GPU exception is the one that catches people migrating a long local job to the cloud: attaching a GPU cuts the maximum task duration from seven days to one hour, so a GPU job has to be sharded into pieces that fit inside an hour each. That is a design constraint, not a setting, and it is better discovered now than at minute sixty-one.