Deploying a Cloud Functions Gen2 Endpoint for a Model Call

A 2nd gen Cloud Function is a Cloud Run service that Google builds from your source. Almost everything that makes it a better host for a model call than a 1st gen function follows from that one substitution, and almost every surprise does too.

What 2nd gen changed underneath

Google’s version comparison page describes a 2nd gen function as a Cloud Run service deployed from source code, where 1st gen functions ran on Google-internal infrastructure. That is not a packaging detail. It means the thing you deploy has a revision, a URL, a concurrency setting, a minimum-instance setting and a request timeout, all of which are Cloud Run concepts, and all of which you can inspect with gcloud run services describe even though you deployed with gcloud functions deploy.

For a function whose whole job is to call a model, this matters more than for a typical webhook, because model calls are slow, bursty, and almost entirely spent waiting on a socket. The 1st gen model — one request per instance, nine-minute ceiling — is close to the worst possible fit for that shape of work.

It also changes what the deploy command does. There is no function archive uploaded to a runtime somewhere; Cloud Build takes your source, runs a buildpack over it, produces a container image, pushes that image to Artifact Registry, and creates a Cloud Run revision from it. Every deploy is a build, which is why the first one takes minutes rather than seconds, and why a broken requirements.txt surfaces as a build log rather than as a runtime error.

One consequence of that is worth acting on before it accumulates. The images land in a repository in your own project — conventionally named gcf-artifacts — and they are not cleaned up for you. A function deployed from CI on every merge produces an image per deploy, each carrying a full Python or Node base layer, and the storage is billed to you. Set a cleanup policy on that repository when you set up the function, not the quarter after somebody notices the line item:

gcloud artifacts repositories describe gcf-artifacts \
  --location=us-central1 --format='value(sizeBytes)'

# keep the most recent 5 versions of each image, delete the rest
gcloud artifacts repositories set-cleanup-policies gcf-artifacts \
  --location=us-central1 \
  --policy=cleanup-policy.json

The four limits that move

• Request timeout. Google documents up to 9 minutes for 1st gen, and up to 60 minutes for 2nd gen HTTP functions. Event-driven 2nd gen functions stay at the lower ceiling.
• Memory and CPU. Up to 8 GB with 2 vCPU on 1st gen; up to 16 GiB with 4 vCPU on 2nd gen.
• Concurrency. One concurrent request per instance on 1st gen; up to 1,000 per instance on 2nd gen.
• Instance floor. 2nd gen exposes a configurable minimum instance count, which 1st gen did not.

The function

This is a Python HTTP function using the Functions Framework, calling a Gemini model through Vertex AI with the function’s own service account. There is no API key anywhere in it, which is the whole point of running inside the project: Application Default Credentials pick up the runtime service account and the model call is authorised by IAM.

# main.py
import os
import functions_framework
from google import genai
from google.genai import types

PROJECT = os.environ["GOOGLE_CLOUD_PROJECT"]
LOCATION = os.environ.get("VERTEX_LOCATION", "us-central1")

# Built once per instance, reused across every request that instance
# serves. With concurrency > 1 this client is shared, so it must be
# safe to use from multiple requests at once — this one is.
client = genai.Client(vertexai=True, project=PROJECT, location=LOCATION)


@functions_framework.http
def summarise(request):
    payload = request.get_json(silent=True) or {}
    text = payload.get("text")
    if not text:
        return ({"error": "field 'text' is required"}, 400)

    response = client.models.generate_content(
        model="gemini-2.5-flash",
        contents=f"Summarise the following in two sentences:\n\n{text}",
        config=types.GenerateContentConfig(
            max_output_tokens=200,
            temperature=0.2,
        ),
    )
    return ({"summary": response.text}, 200)

# requirements.txt
functions-framework==3.*
google-genai

The client construction sits at module scope deliberately. Module-level work runs once per instance rather than once per request, and a client that builds a credential chain and a connection pool is exactly the work you want amortised. It is also the code most likely to raise during a cold start, which is worth remembering when a deploy fails with a startup error rather than a request error.

Deploying it

1. Enable the APIs the deploy path needs. A 2nd gen deploy builds an image, so Cloud Build and Artifact Registry are involved as well as the function and run APIs: gcloud services enable cloudfunctions.googleapis.com run.googleapis.com cloudbuild.googleapis.com artifactregistry.googleapis.com aiplatform.googleapis.com.
2. Create a dedicated runtime service account rather than using the default compute one, and grant it only roles/aiplatform.user. Why that role and not a broader one is the subject of what each Vertex AI IAM role actually grants.
3. Deploy from the source directory, pinning the generation explicitly:

   gcloud functions deploy summarise \
     --gen2 \
     --runtime=python312 \
     --region=us-central1 \
     --source=. \
     --entry-point=summarise \
     --trigger-http \
     --no-allow-unauthenticated \
     --service-account=fn-summarise@PROJECT_ID.iam.gserviceaccount.com \
     --memory=512Mi \
     --cpu=1 \
     --timeout=120s \
     --concurrency=8 \
     --max-instances=20 \
     --set-env-vars=VERTEX_LOCATION=us-central1

4. Call it with an identity token, because --no-allow-unauthenticated means the endpoint requires an IAM-authenticated caller:

   URL=$(gcloud functions describe summarise --region=us-central1 \
     --gen2 --format='value(serviceConfig.uri)')
   
   curl -sS -X POST "$URL" \
     -H "Authorization: Bearer $(gcloud auth print-identity-token)" \
     -H "Content-Type: application/json" \
     -d '{"text":"Cloud Run charges for CPU and memory allocated to an instance."}'

Concurrency is the setting that surprises people

--concurrency=8 in that deploy is the setting most 2nd gen tutorials leave at its default and never mention, and for a model-calling function it is the setting that decides your bill.

A request that spends 3 seconds waiting on a Vertex response is using almost no CPU for those 3 seconds. At concurrency 1 you pay for a whole instance to sit idle through each of them and you scale out one instance per concurrent request. At concurrency 8, one instance covers eight simultaneous waits and you pay for roughly one eighth as much allocated capacity for the same traffic. This is the single largest cost lever on the function and it costs nothing to set.

The counterweight is that everything at module scope is now genuinely shared. A client that is not safe to use concurrently, a global accumulator, a cached credential written back to a module variable — all become race conditions that did not exist at concurrency 1. And because CPU is shared too, a function that does real local work between model calls will start queueing on itself. The honest rule is that concurrency should be high when the function is mostly waiting and low when it is mostly computing; nothing about the platform decides that for you. The wider argument, including where the CPU actually goes, is in concurrency settings for inference services on Cloud Run.

Verifying and tearing down

Confirm the Cloud Run heritage directly — it is the fastest way to see that a setting you made with the functions CLI landed where you think:

gcloud run services describe summarise --region=us-central1 \
  --format='yaml(spec.template.spec.containerConcurrency,
                 spec.template.spec.timeoutSeconds,
                 spec.template.metadata.annotations)'

gcloud functions delete summarise --region=us-central1 --gen2 --quiet

If the concurrency and timeout you set with gcloud functions deploy show up in that Cloud Run description, you have confirmed the substitution is real, and you can reason about the function as a Cloud Run service from then on — including for the timeout behaviour covered in fixing a Cloud Functions timeout on a slow model call.

If you are converting an existing 1st gen function rather than writing a new one, plan the cutover rather than editing a flag. The two generations are separate resources with separate URLs, and the --gen2 flag on a deploy is not an in-place upgrade: deploying the new one under the old name in the same region conflicts, and every caller holding the old URL keeps hitting the old function until you move it. The workable sequence is to deploy the 2nd gen function under a new name, move callers to the new URL, watch both sets of logs until the old one goes quiet, and only then delete it. Budget for the fact that the handler signature changes too — the 1st gen background function signature and the CloudEvent one are not the same shape, so an event-driven function is a rewrite of its entry point rather than a redeploy.