Running Text Generation on Cloudflare Workers AI

A Workers AI text-generation call is three lines of code and one line of configuration. The part worth getting right is what you do with the object that comes back, because it carries the token counts you will later be billed on and it changes shape the moment you turn streaming on.

The binding, not an API key

Workers AI is reached through a runtime binding rather than an HTTP client with a secret. You declare it in your Wrangler configuration and the runtime injects it into env. There is no key to rotate, no base URL to hard-code, and no credential in your bundle — the request is authenticated by the fact that it originated inside your Worker.

// wrangler.jsonc
{
  "name": "text-gen",
  "main": "src/index.ts",
  "compatibility_date": "2026-08-01",
  "ai": {
    "binding": "AI"
  }
}

The binding value is the property name on env, so the example above gives you env.AI. Cloudflare’s Workers AI getting-started guide uses exactly this shape; the equivalent in the older TOML format is an [ai] table with binding = "AI". Run npx wrangler types after adding it and the generated Env interface will carry the binding, which is the cheapest way to catch a typo in the property name before it becomes a runtime undefined.

One call, end to end

1. Create the project with npm create cloudflare@latest and pick the Hello World Worker template, or add the ai block above to a Worker you already have.
2. Add the binding to wrangler.jsonc exactly as shown, then regenerate types.
3. Replace the fetch handler with the code below. It reads a query parameter so you can change the prompt without redeploying.
4. Run npx wrangler dev. Local development still calls the real models over the network — there is no local inference — so you need to be logged in with npx wrangler login first.
5. Deploy with npx wrangler deploy and hit the workers.dev URL with ?q= and a question.

// src/index.ts
export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    const url = new URL(request.url);
    const question = url.searchParams.get("q") ?? "Why is the sky blue?";

    const result = await env.AI.run("@cf/meta/llama-3.1-8b-instruct", {
      messages: [
        { role: "system", content: "Answer in two sentences. No preamble." },
        { role: "user", content: question },
      ],
      max_tokens: 256,
    });

    return Response.json(result);
  },
} satisfies ExportedHandler<Env>;

env.AI.run() takes the model id first, the model inputs second and an options object third. The model id is the full slug including the publisher segment — @cf/meta/llama-3.1-8b-instruct, not llama-3.1-8b-instruct. Getting that wrong returns error 3042, documented by Cloudflare with HTTP 404 and the message “The model name is invalid”, which is a much friendlier failure than the silent 200-with-nonsense some gateways produce.

What comes back

For a non-streaming text-generation call the result is a plain object, and Cloudflare’s model schema documents three fields on it:

• response — a string, the generated text. This is the one people use and the one that disappears when you enable streaming.
• usage — an object with prompt_tokens, completion_tokens and total_tokens. These are the numbers the neuron charge is computed from, so logging them per request is how you attribute cost later without guessing from character counts.
• tool_calls — an array, populated only when you passed a tools definition and the model chose to call one.

Do not write result.response.trim() without a guard. A call that hits a stop condition before emitting anything, or a model that decided to emit a tool call instead of prose, will leave you dereferencing a property that is not there. The defensive read is cheap:

const text = typeof result.response === "string" ? result.response : "";
if (!text && result.tool_calls?.length) {
  // the model wanted a tool, not an answer
}
console.log("tokens", result.usage?.total_tokens);

prompt or messages, and why it matters

Text-generation models on Workers AI accept either a bare prompt string or a messages array of { role, content } objects. Cloudflare’s own quickstart uses prompt because it is shorter. Prefer messages in anything you intend to keep.

The reason is that a chat-tuned model has a template — a specific arrangement of role markers it was trained to see — and the messages form lets the runtime apply that template for you. Passing prompt hands the model raw text, and on an instruct model that usually still works, but you lose the ability to separate a system instruction from user input. That separation is not cosmetic: it is the only structural signal the model has that the user’s text is data rather than instructions, and it is the difference between a prompt injection being awkward and being trivial.

The two forms are mutually exclusive. Send both and behaviour is undefined in the useful sense — you are relying on which key the schema validator reads first. Pick one.

max_tokens deserves a deliberate value rather than the default, for two reasons that pull in the same direction. It is a hard stop on the most expensive half of the request, since output tokens are charged at several times the input rate on every model in Cloudflare’s table. And it is the only bound you have on how long the call takes, because generation time scales with the number of tokens produced. A generous max_tokens on an endpoint a human is waiting for is a latency decision disguised as a safety valve.

Be aware that hitting the cap truncates mid-sentence rather than producing a shorter well-formed answer. If the response is parsed — JSON, a list, anything structured — a truncation is a parse failure rather than a partial result, so either leave enough headroom that it never fires, or treat a response that reached the cap as an error rather than as data.

What this costs you in CPU time

The most common surprise here is a non-problem that people spend hours on. Cloudflare’s Workers limits page documents a CPU time budget of 10 ms per invocation on the Free plan and 30 seconds by default on Paid, configurable up to five minutes. A model call can easily take longer than 10 ms of wall-clock time, and it does not matter, because CPU time is time spent executing your code — not time spent awaiting a network response. Awaiting env.AI.run() for twenty seconds consumes essentially no CPU budget.

What does consume it is what you do afterwards: parsing a large JSON body, running a regular expression over a long completion, or decoding base64. That asymmetry is worth internalising because it tells you where to look when a Worker really does exceed its budget — never at the model call, always at the post-processing. The same page documents 128 MB of memory per isolate, which is the other ceiling post-processing runs into first.

Once this works, the two natural next steps are turning the same call into a stream so the reader sees tokens as they arrive and understanding what those usage counters translate into in neurons.