Command R’s Grounded and Ungrounded Generation Modes

Searching for the flag that turns on grounded generation in Cohere’s API returns nothing, because there is no flag. Grounded mode is entered by supplying documents on the request. The model then answers from them and returns a citation structure alongside the text, mapping spans of its answer back to the documents they came from.

There is no grounded flag

The Command family — Command R, Command R+, and the Command A line — was trained specifically for retrieval-augmented generation, and Cohere exposes that as a shape of request rather than as a mode setting. Send no documents and you get an ordinary chat completion. Send documents and the model switches into grounded generation: it is trained to answer from what you supplied and to mark which parts of its answer came from where.

Why build it that way rather than as a switch? Because the two things the switch would have to coordinate are already implied by the request. A grounded answer needs sources to be grounded in, and citations need identifiers to point at. Both come from documents. A boolean that could be set to true with no documents present would have no defined meaning, and one that could be set to false with documents present would be asking the model to ignore what you just sent it. The presence test is the only combination that is always coherent.

This is worth stating clearly because it changes what you go looking for. The knobs that do exist — citation_options, search_queries_only, prompt_truncation — all tune grounded generation. None of them enables it. The enabling is structural.

Ungrounded: the plain request

curl https://api.cohere.com/v2/chat \
  -H "Authorization: Bearer $CO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "command-r-plus",
    "messages": [
      {"role": "user", "content": "What is our refund window?"}
    ]
  }'

The response carries a message with content and nothing else of interest:

{
  "id": "...",
  "message": {
    "role": "assistant",
    "content": [
      {"type": "text", "text": "Refund windows vary by retailer. Commonly ..."}
    ]
  },
  "finish_reason": "COMPLETE",
  "usage": { ... }
}

Note what the model did with a question it had no basis to answer: it answered generally. There is no citations array, because there is nothing to cite. Anything the model says here comes from its weights, and you have no way to tell which parts are grounded in anything at all. That is the property grounded mode exists to remove.

Grounded: documents in, citations out

Add a documents array. Each document is an object of your own fields — Cohere does not impose a schema beyond an optional id, and the field names you choose are visible to the model, so call them something meaningful.

curl https://api.cohere.com/v2/chat \
  -H "Authorization: Bearer $CO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "command-r-plus",
    "messages": [
      {"role": "user", "content": "What is our refund window?"}
    ],
    "documents": [
      {
        "id": "policy-14",
        "data": {
          "title": "Returns policy",
          "text": "Customers may return unopened items within 30 days of delivery for a full refund. Opened items are eligible for store credit within 14 days."
        }
      },
      {
        "id": "faq-3",
        "data": {
          "title": "Shipping FAQ",
          "text": "Delivery takes 3 to 5 working days within the UK."
        }
      }
    ]
  }'

The response now carries citations:

{
  "message": {
    "role": "assistant",
    "content": [{
      "type": "text",
      "text": "Unopened items can be returned within 30 days of delivery for a full refund, and opened items are eligible for store credit within 14 days."
    }],
    "citations": [
      {
        "start": 0,
        "end": 62,
        "text": "Unopened items can be returned within 30 days of delivery",
        "sources": [
          {"type": "document", "id": "policy-14", "document": { ... }}
        ]
      },
      {
        "start": 68,
        "end": 145,
        "text": "opened items are eligible for store credit within 14 days",
        "sources": [
          {"type": "document", "id": "policy-14", "document": { ... }}
        ]
      }
    ]
  },
  "finish_reason": "COMPLETE"
}

The second document was supplied and not cited, which is the correct behaviour and worth checking for in your own testing. Grounded mode does not mean the model uses everything you give it; it means what it does use is traceable.

The document shape is worth one more note. Cohere does not impose a schema on data, and the keys you choose are rendered into the model’s view of the document. So title and text read as what they are, whereas f1 and f2 read as nothing, and the difference shows up in how well the model uses the fields. Include the metadata that would help a person judge the source — a date, a section heading, a URL — and leave out anything the model should not be quoting back, because everything in the object is visible to it.

Reading the citation objects

Each citation has four things you need, and the first two are the ones people get wrong.

• start and end — character offsets into the generated text, not token indices and not offsets into your document. They are what you use to render a footnote marker or a highlight in the right place.
• text — the substring those offsets delimit. Redundant with the offsets, and worth using as a consistency check: if slicing your own copy of the text at those offsets does not give you this string, you have an encoding mismatch. This is a real hazard, because offsets computed over a UTF-8 byte sequence do not match offsets over a JavaScript UTF-16 string once any character outside the basic multilingual plane is present.
• sources — which documents that span came from. Plural: a sentence synthesised from two documents cites both.
• Coverage is partial by design. Not every character of the answer is inside a citation span. Connective text and the model’s own phrasing are uncited, and an uncited span is not an error. Your renderer has to handle gaps.

The same structure appears when grounding comes from tool results rather than from a documents array — see how tool results are shaped and the citations response in detail. In streaming, citations arrive as their own event type rather than inside the text deltas, so a client that only handles content events will drop them silently. That is covered in Cohere’s streaming event types.

Tuning the citation behaviour

Once you are in grounded mode there are three settings that change how it behaves.

• citation_options.mode — Cohere documents a fast mode and an accurate mode, plus turning citations off entirely. The trade is latency against how carefully the spans are attributed. Off is worth knowing about: it keeps grounded generation’s answer-from-these-documents behaviour while dropping the citation pass, which is what you want when you are grounding for accuracy and not displaying sources.
• search_queries_only — returns the search queries the model would issue for this turn and generates no answer. This is the first half of a retrieval loop: ask what to look for, run your own retrieval, then send the results back as documents.
• prompt_truncation — decides what happens when your documents plus history exceed the context window. Automatic truncation drops the least relevant documents; the alternative is an error. Which you want depends on whether a silently shortened context is worse for you than a failed request, and for anything auditable it usually is.

Those three interact in a way worth planning for. If you are running a retrieval loop with search_queries_only, you have two round trips per user turn before any answer is generated, and the second one carries your whole retrieved set. That is where prompt_truncation starts mattering, because a retrieval step that returns twenty documents will overflow a context that ten fitted in comfortably. And it is where the citation mode’s latency cost lands too, on the turn the user is actually waiting on. Decide the three together rather than one at a time.

One last thing that is easy to assume and wrong: grounding is not a guarantee of factuality. The model is trained to answer from the documents and to attribute what it uses, and it does that well, but a citation is a claim that a span came from a source, not a check that the span is a faithful restatement of it. If correctness matters, the citation offsets give you the means to verify — which is more than an ungrounded answer offers, and less than a proof.