Fixing a Cloud Functions Timeout on a Slow Model Call

A model call that takes 90 seconds can be killed by three different deadlines, and they produce three different symptoms. Identify which one fired before changing any setting, because two of the three are not in the function’s configuration at all.

What you actually saw

On a 2nd gen HTTP function, exceeding the request timeout closes the connection and returns a 504. Google’s Cloud Run documentation describes the behaviour: if a response is not returned within the configured time, the network connection is closed and a 504 is returned. Depending on where the request entered — direct, or through a load balancer — the body is typically an upstream request timeout page rather than anything your code produced. On 1st gen, the log line is the familiar Function execution took N ms, finished with status: ‘timeout’.

One detail from Google’s request timeout documentation is easy to miss and explains a lot of confusing behaviour afterwards: the instance that served the timed-out request is not terminated. Your code may still be running, still holding the outbound HTTP connection, still about to write to a database, while the caller has already received a 504 and possibly retried. A timeout is not a rollback.

Three clocks, not one

• The function’s request timeout. Set with --timeout. Google documents up to 60 minutes for 2nd gen HTTP functions and up to 9 minutes for 1st gen; event-driven 2nd gen functions have a lower ceiling than HTTP ones. Fires as a 504 with nothing from your code in the response.
• The outbound call’s own deadline. Set by the SDK or HTTP client you use to reach the model. Fires as an exception inside your handler, which means you see a stack trace in your own logs and you get to choose the status code.
• The caller’s deadline. A browser, an API gateway, a load balancer backend service, or another function. Fires with no trace of it in your logs at all, because from the function’s side the request simply completes into a closed socket.

The test that separates them takes one look at the logs. If your handler logged an exception, it was the outbound clock. If the platform logged the request with a 504 and your handler logged nothing at the end, it was the function timeout. If your handler logged a successful completion and the user still saw an error, it was the caller’s clock and no amount of --timeout will help.

Raising the function timeout

If it is genuinely the function clock, raise it and set a maximum instance count at the same time. A long timeout without an instance ceiling is how a slow dependency turns into a large bill: instances accumulate because none of them finish, and Cloud Run keeps starting more.

gcloud functions deploy summarise \
  --gen2 --region=us-central1 \
  --timeout=300s \
  --max-instances=20 \
  --concurrency=8

Raise it to a number you chose, not to the ceiling. The timeout is your only backstop against a hung dependency, and setting it to 60 minutes because 60 minutes is allowed means a stuck request holds capacity for an hour. A model call that normally takes 8 seconds and is allowed 300 is already generous by a factor of thirty.

The outbound call has its own deadline

This is the clock people forget, and it is the one you should usually be setting rather than the function timeout. If the model call has no explicit deadline, it inherits whatever the SDK’s default is, and that default is frequently longer than your function timeout — which produces exactly the failure mode above, where the platform kills the request and your code never gets a chance to log why.

from google.genai import types

# Client-level timeout, in milliseconds. Set it below the function's
# own timeout so the exception happens inside your handler, where you
# can log it, count it, and return something the caller understands.
client = genai.Client(
    vertexai=True,
    project=PROJECT,
    location=LOCATION,
    http_options=types.HttpOptions(timeout=60_000),
)

The ordering rule is worth stating plainly, because it is the whole fix in one line: caller timeout > function timeout > outbound timeout. Set them in that order with real gaps between them and every failure surfaces at the innermost layer that knows what happened, which is the only layer that can log a useful message or fall back to a second model.

When more time is the wrong fix

Some of these should not be synchronous requests at all. If the work reliably takes minutes rather than seconds — a long document summarised in passes, a batch scored row by row — a longer timeout is holding an HTTP connection open for the convenience of not building a queue.

Three alternatives, in roughly increasing order of effort. Stream the response, which does not make the work faster but restarts the clock on the caller’s side because bytes are flowing; see streaming a response from Cloud Run. Accept the request, return a 202 with a job ID, and do the work in a Cloud Run job or a Pub/Sub-triggered worker — Cloud Run jobs for batch inference is the shape of that. Or split the work so each request does one pass and the caller drives the loop, which turns one 300-second unit of risk into ten 30-second ones that can each be retried independently.

The last of those is usually the right answer for model work specifically, because a retried 30-second call costs 30 seconds of tokens and a retried 300-second call costs 300. Retry cost is the part of the timeout decision that has nothing to do with latency, and it is covered on its own in what retries actually cost.