What Function Calling Gemma Supports Natively

Gemma has no native tool-call format in the sense that some other families do. There are no reserved tool tokens in the vocabulary and no tool branch in the chat template, at any generation. Gemma 3 is documented as capable of function calling, and that capability is a prompting pattern rather than a protocol.

The short answer, and how to verify it

Some model families ship dedicated machinery: a role for tool results, reserved tokens that open and close a call, a tools argument that the bundled template renders into a documented block. Gemma ships none of that. Its template accepts a list of messages with two roles and renders them into the start-of-turn format.

You do not have to take that on trust. Print the template and look:

from transformers import AutoTokenizer

for repo in ["google/gemma-2-9b-it", "google/gemma-3-4b-it"]:
    tok = AutoTokenizer.from_pretrained(repo)
    t = tok.chat_template or ""
    print(repo, "mentions tools:", "tools" in t, "| tool_calls:", "tool_calls" in t)

A template that supports native tool calling has to reference the tool list somewhere, because it must render it. If the string does not mention it, the checkpoint has no opinion about how tools are formatted, and every framework that offers “Gemma function calling” is supplying a convention of its own on top.

What the template does not contain

Three absences follow, and each one is work you inherit.

• No tool-definition rendering. Nothing turns a JSON Schema into the text the model sees. You decide the format and you write it into the prompt.
• No delimiter around a call. There is no token pair equivalent to a start and end of tool call, so a call is ordinary model output that you have to recognise by pattern.
• No tool role for results. A tool result comes back as a user turn, because that is the only inbound role that exists.

What Google does provide is guidance. The Gemma documentation at ai.google.dev/gemma/docs describes function calling with Gemma 3 as a prompting technique and gives a worked convention. Treat that as a recommended recipe rather than as a wire format the weights enforce.

The distinction between a convention and a trained format is not academic, and it predicts the behaviour you will see. A model trained with tool tokens has the call boundary in its vocabulary: it is structurally hard for it to half-emit one, and a constrained decoder can force the shape almost for free. A model that produces a call as ordinary text can also produce something that looks almost like a call, explain the call in prose instead of making it, or wrap it in an apology. All three are things Gemma does occasionally, and none of them is a bug. They are what “prompted convention” means in practice, and your parser has to expect them.

The upside of owning the format is that you are not constrained by somebody else’s schema. If your tools take one string argument, you can define a one-line call syntax that a 2B model emits far more reliably than nested JSON. That freedom is genuinely worth something at these sizes, where the difference between a format the model can hold and one it cannot is the difference between a working feature and a flaky one.

Defining a convention that works

Because you own the format, pick one that is easy to detect and hard to emit by accident. A fenced block with an explicit tag beats bare JSON, since bare JSON is something a model produces for many innocent reasons.

SYSTEM = """You have access to these tools:

get_weather(city: string, unit: "c" | "f") -> {"temp_c": number, "conditions": string}
get_time(timezone: string) -> {"iso": string}

To call a tool, reply with nothing but a fenced block:

```tool_call
{"name": "get_weather", "arguments": {"city": "Lisbon", "unit": "c"}}
```

Call at most one tool per turn. If no tool is needed, answer normally."""

Because Gemma has no system role before Gemma 3, that text is folded into the first user turn, exactly as the system-role page describes. On Gemma 3 you may pass it as a system message and the template performs the same fold for you.

Parsing is then a matter of looking for your own fence:

import json, re

FENCE = re.compile(r"```tool_call\s*(\{.*?\})\s*```", re.S)

def parse_tool_call(text):
    m = FENCE.search(text)
    if not m:
        return None
    try:
        call = json.loads(m.group(1))
    except json.JSONDecodeError:
        return None                      # treat as prose, or retry once
    if "name" not in call:
        return None
    call.setdefault("arguments", {})
    return call

Returning None on a malformed block rather than raising is deliberate. A small model will occasionally emit a nearly-correct block, and the recovery you want is usually one retry with the parse error appended, not a crashed request.

Getting the result back in

With no tool role, the result goes back as a user turn, labelled clearly enough that the model does not mistake it for a human speaking:

messages.append({"role": "model", "content": raw_model_output})
messages.append({
    "role": "user",
    "content": "Tool result for get_weather:\n" + json.dumps(result),
})

Append the model’s own call verbatim before the result. If you drop it and show only the result, the transcript contains an answer to a question the model cannot see it asked, which is a reliable way to get a confused second turn.

Because the result arrives as a user turn, roles must still alternate, so two tool calls in a row need their results merged into a single user turn rather than appended as two. And errors should be reported in the same channel rather than swallowed: a turn reading “Tool result for get_weather: error, unknown city” lets the model correct itself, whereas silently retrying leaves it repeating the same wrong call.

Making it robust at 2B and 4B

• Keep the tool list short. Instruction-following degrades with the number of options at these sizes far faster than it does on frontier models. Five tools is a different problem from twenty-five.
• Constrain decoding if your stack allows it. A grammar or JSON-schema-constrained sampler removes the entire class of malformed-call failures, and is more effective than any amount of prompt wording.
• Give one worked example. A single correct call in the prompt is worth several paragraphs of instruction, and costs fewer tokens than the paragraphs.
• Validate arguments against the schema before executing. The model is producing text that resembles a call. Nothing checked the types.

Architecturally, the pattern that survives contact with a small model is one call per turn with an explicit loop around it, rather than a plan that emits several calls at once. Parallel tool calls require the model to hold multiple argument sets correctly in one output, and the error rate compounds. A loop that makes one call, feeds the result back and asks again is slower in wall-clock terms and dramatically more reliable, and it also gives you a natural place to stop after a fixed number of iterations rather than discovering a runaway agent in your bill.

It is also worth deciding up front what happens when the model answers in prose instead of calling a tool it should have called. Retrying with a firmer instruction works sometimes; a better default in a production path is to detect the missing call, fall back to a deterministic branch, and log the case. A small model used as a router is most valuable when the surrounding code assumes it will occasionally decline to route.