Vision Input in Grok: Image Support and Token Cost

xAI documents four hard facts about image input and does not document a fifth that everybody wants: what an image costs in tokens. The API reports that number back to you on every request, which turns out to be a better answer than a formula.

The documented limits

From xAI’s image understanding guide, at the time of writing:

maximum image size    20 MiB
accepted formats      jpg / jpeg, png
images per request    no limit
ordering              any image/text input order is accepted
detail                a "detail" field is available (example uses "high")

Three of those deserve a sentence each. 20 MiB is mebibytes — 20 × 1024 × 1024 = 20,971,520 bytes — and it is the size of the encoded image, so if you are sending base64 it is worth checking which side of the encoding your check is on, since base64 inflates by about a third.

JPEG and PNG only is a narrower list than several competing APIs accept. WebP, GIF and HEIC are not on it. If your uploads come from browsers or phones you will receive all three, so transcoding to PNG or JPEG belongs in your ingest path rather than in an error handler.

No documented limit on image count is unusual — most providers publish a per-request cap, and some cap it quite low. “No limit” here means no separate count limit, not free: every image is prompt tokens, and the real ceiling is the context window and the request size.

There is no published token formula

Some vendors publish the arithmetic — a base cost plus a per-tile cost, with a documented tiling rule, so you can compute an image’s token count from its dimensions before sending it. xAI does not. Its image understanding guide gives the constraints above and the request shape, and does not state how image tokens are derived from pixel dimensions, nor how the detail field changes that derivation.

This page therefore does not give you a formula, and you should be suspicious of any page that gives you one for Grok while citing xAI. Formulas for this get copied between vendors and quietly mis-attributed — the numbers look authoritative, they are arithmetically consistent, and they are somebody else’s. A wrong formula is worse than no formula, because it is used with confidence.

The field that answers it

The usage object breaks the prompt down by modality. In the response shape xAI documents for streaming, and in the non-streaming response, the breakdown appears as:

"usage": {
  "prompt_tokens": 1483,
  "completion_tokens": 226,
  "total_tokens": 1709,
  "prompt_tokens_details": {
    "text_tokens": 1102,
    "audio_tokens": 0,
    "image_tokens": 381,
    "cached_tokens": 0
  }
}

image_tokens is the number. It is reported per request, for the model that served it, by the system that bills you — which makes it strictly better evidence than a published formula would be, because a formula can go out of date while a response field cannot.

The measurement that answers your budgeting question takes about five minutes and no benchmarking: send the same short text prompt with no image and read text_tokens; send it again with one image at your typical dimensions and read image_tokens; repeat at a couple of sizes and at each detail setting you intend to use. That gives you the cost of your images on your model, which is the only number that goes into a real budget. Do it per model — there is no documented guarantee that two Grok versions count an image the same way.

If you want streamed responses to include this, you must ask for it: send stream_options with include_usage: true, or the usage chunk is never emitted. See the streaming format page.

Sending an image

Image content goes in the message content array in the OpenAI-compatible shape, either as a URL or as a data URI. Text and images may be interleaved in any order.

{
  "model": "grok-4.5",
  "messages": [
    {
      "role": "user",
      "content": [
        { "type": "text", "text": "Which of these two invoices has the later due date?" },
        { "type": "image_url",
          "image_url": { "url": "https://example.com/invoice-a.png", "detail": "high" } },
        { "type": "image_url",
          "image_url": { "url": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAA...",
                         "detail": "high" } }
      ]
    }
  ],
  "max_completion_tokens": 300
}

Both grok-4.5 and grok-4.3 are documented as taking text and image input and producing text output. There is no image output from these models; xAI’s image generation is a separate family of grok-imagine-* models with per-image pricing rather than per-token.

URL or data URI

The two transports are not interchangeable, and the choice has consequences that only appear in production.

A URL means xAI’s servers fetch the image. That keeps your request small and your upload bandwidth free, and it imposes three requirements people meet by accident and then break: the URL must be reachable from xAI’s infrastructure, not from your network; it must be reachable without your credentials, which rules out a signed session and argues for a short-lived pre-signed link; and the fetch is a network round trip on the critical path of your request, so a slow origin is added directly to your time to first token. An image behind a VPN, on localhost, or on a host that blocks unfamiliar user agents simply does not arrive.

A data URI means you carry the bytes. Nothing can fail to be fetched, nothing leaks a URL, and the image never needs to be publicly addressable — which for anything containing personal data is usually the deciding argument. The cost is request size, and this is where the 20 MiB cap needs care. Base64 inflates by four bytes per three, about 33%. If the documented cap is applied to what you transmit, a 20 MiB budget holds roughly 15 MiB of original image:

20 MiB          = 20 x 1024 x 1024 = 20,971,520 bytes
base64 overhead = 4 bytes out for every 3 in  (~1.333x)
source budget  ~= 20,971,520 / 1.333       ~= 15.7 MiB

That derivation assumes the limit is measured on the encoded payload, which xAI does not state either way — so treat 15 MiB as the safe working ceiling for a data URI rather than as a documented figure. Since the resolution that actually helps a model read a document is almost always well under either number, the honest advice is to stop optimising against the cap and downscale on ingest.

Budgeting without a formula

Three practices cover most of what the missing formula would have given you.

• Downscale before sending. Whatever the counting rule, more pixels do not cost fewer tokens. Send the smallest image at which the task is still answerable — for reading a document that is usually far smaller than the camera produced.
• Treat image count as a budget line. “No limit” makes it easy to attach a folder of screenshots to one request. Several hundred image tokens each adds up quickly against the 200k threshold where the per-token rate doubles, and images are the least visible way to cross it.
• Alert on image_tokens, not on image count. The count is what you control; the tokens are what you pay. Log the field and you will see a dimension change in your upload pipeline the week it happens rather than at the end of the month.