Cloud Run Cold Starts and Container Image Size

The standard advice for Cloud Run cold starts is to shrink the container image. Google’s own documentation says that will not help: because of container image streaming, the size of the image does not affect container startup time. Understanding why turns a wasted optimisation into the two or three that do apply.

What a cold start is made of

When a request arrives and no instance is available to serve it, four things have to happen before the request is handled. A sandbox is allocated. The container filesystem is made available. The entrypoint runs and the process initializes. The process starts listening on the configured port, at which point Cloud Run routes the request to it.

On most container platforms the second step is a download: pull every layer of the image from the registry, decompress it, and assemble a filesystem before anything can execute. That step scales with image size, which is where the shrink-the-image advice comes from, and it is correct on platforms that work that way. Cloud Run does not work that way.

Image streaming, and what it removes

Google’s general development tips for Cloud Run state it directly: “Because of Cloud Run’s container image streaming technology, the size of your container image does not affect container startup times or request processing time.” The same page adds that the image size does not count towards the available memory of the container. Google documents both claims on that page.

The mechanism is lazy loading. Rather than downloading the whole image and then starting the process, the platform presents the image as a filesystem backed by remote content, and fetches blocks when something reads them. A 6 GB image whose startup path touches 300 MB fetches roughly 300 MB before serving. The other 5.7 GB is never read and never paid for in startup latency.

This changes what an optimisation means. Under a pull-then-start model, the number that matters is the total compressed size of the image. Under streaming, the number that matters is how much of the image the startup path reads — which is a completely different quantity, and one that a smaller image does not automatically reduce. Deleting 200 MB of build tooling that was never read at runtime changes nothing about startup. Reordering an import so the process loads a large library lazily instead of at module scope changes a great deal.

Where image size still costs you

None of this makes a large image free. It relocates the cost.

• Build and push time. Every CI run pays for it, and a large image makes the feedback loop on a one-line change unpleasant. This is the cost people actually feel most often.
• Registry storage and egress. Charged per gigabyte, per copy, per region you replicate to.
• Vulnerability surface. Google’s own recommendation on the same page is to keep images lean for security reasons — fewer packages, fewer findings — using lightweight bases and multi-stage builds. That argument is intact and is the honest reason to do the work.
• The first read of a cold block. Lazy loading means some reads stall on a fetch. If a code path is exercised for the first time twenty seconds into serving, that fetch happens then, and appears as a slow request rather than as a slow start. The total is not larger; it is smeared.

The levers that do move the number

Google names three, and they attack different parts of the four-stage sequence above.

• Startup CPU boost. Cloud Run allocates extra CPU during instance startup and for 10 seconds after the instance has started — Google’s documentation gives the example of a 1 vCPU instance boosting to 2 vCPUs for that window. Since the expensive part of most inference container startups is CPU-bound initialization, this is usually the highest-yield setting available, and it is free when no instance is starting. Set it with --cpu-boost on gcloud run deploy.
• Minimum instances. Keeping instances provisioned removes the cold start entirely for as much traffic as they cover, at the cost of paying for idle capacity. This is the only lever that makes the p99 cold start disappear rather than shrink.
• Lazy initialization. Deferring the construction of things not needed on the first request moves work out of startup, at the cost of a slower first request that needs them. Google documents this trade explicitly. For an inference proxy this often means not building every provider’s client at import time.

A fourth belongs on the list even though it is about the request rather than the instance: concurrency. Cloud Run serves multiple requests per instance, so raising concurrency means fewer instances for the same traffic and therefore fewer cold starts in total. For a service that spends its time awaiting a model API rather than computing, the default is often far below what the process could handle.

The model weights question

Everything above concerns the image. Model weights are a separate decision, and it is the one that actually dominates cold start for a self-hosted model on Cloud Run: baked into the image, they are streamed like everything else, but the startup path reads all of them, so the whole file is fetched before the server is ready. Streaming does not help a file you read in full.

Google’s own guidance on AI cold starts on Cloud Run is that baking weights into the image is efficient for smaller models thanks to image streaming, while for very large models the import and streaming overhead becomes a bottleneck — with a threshold in the region of ten gigabytes, on the Google Cloud blog’s AI cold starts guide. Treat that as a starting point for your own measurement rather than a constant: it depends on the model, the machine type and how the loader reads the file.

The alternative — fetching weights from Cloud Storage at startup — moves the same bytes over a different path and adds a network hop, so it is not automatically faster. What it does buy is the ability to update weights without rebuilding and redeploying an image, which is often the real reason to do it. Either way, this is the number to instrument first, because it makes the entire image-size discussion a rounding error. The same trade-off in a cluster is the subject of baking versus mounting weights.