Detail Levels: When Low-Res Image Input Is Enough

The detail parameter is usually presented as a quality-versus-cost dial, which is true and useless. It is really a resolution decision, and resolution decisions can be settled with a division: how many pixels will the smallest thing you need to read still have after the provider resizes the image?

A note on what this page is not. There is no accuracy table here, because the honest version of that table would need to be measured on your images and nobody has measured it on them. What follows is the arithmetic that predicts which way the result will go, plus a protocol for getting the number for your own workload in an afternoon.

What the flag does

On the providers that expose it, low detail replaces the tiling step with a single downscaled encoding of the whole image, and charges a flat base cost — on OpenAI’s documented scheme for the GPT-4o generation that is 85 tokens regardless of input size, against 765 for a typical square in high detail. High detail keeps the global view and adds encoder-sized crops so that local regions survive at something near native resolution.

Two facts follow that people routinely get wrong. Low detail is not “a worse model” — it is the same model given less information. And high detail is not a magnifying glass: it will not recover detail that was already below the sensor’s resolution or destroyed by JPEG artefacts in the file you uploaded.

It also helps to know which tasks are genuinely resolution-bound and which are not, because the split is sharper than intuition suggests. Scene-level questions — what kind of room is this, is anyone present, is the photograph indoors, does this look like a receipt or a passport — are answered from the gist and survive aggressive downscaling almost unchanged. Anything requiring a small high-frequency feature to be resolved — a digit, a checkbox state, a serial number, a hairline crack, the tick on an axis — falls off a cliff. Between them sits a band of tasks that degrade gracefully, such as counting a handful of large objects, and it is that middle band where an A/B is worth running rather than reasoned about.

The minimum-feature-size rule

Name the smallest element the task depends on, in pixels of the original image. A digit in a table cell. A checkbox. A serial number etched on a part. Then compute what survives:

surviving_px = feature_px * (target_edge / original_long_edge)

The thresholds worth carrying in your head come from OCR practice rather than from anything about transformers. Latin text needs roughly 20 pixels of glyph height to be read reliably by machine and starts degrading badly under about 10; this is why the classical OCR guidance has been 300 dpi for decades. A checkbox or a filled-in circle needs far less — a dozen pixels across is plenty, because you are detecting presence, not shape. A face for identity-free description needs less still.

Worked: a scanned A4 page

scan at 300 dpi   ->  2480 x 3508 px
body text 10 pt   ->  cap height approx 0.7 * 10/72 inch
                  ->  0.097 in * 300 dpi = 29 px       legible

low detail: long edge squashed toward ~512 px
  scale = 512 / 3508 = 0.146
  29 px * 0.146 = 4.2 px                              gone

high detail: long edge to 2048, short side to 768
  final about 768 x 1086
  scale = 1086 / 3508 = 0.310
  29 px * 0.310 = 9.0 px                              marginal

crop of one table, 900 x 300 px, sent whole
  short side 300 upscaled to 768 -> scale 2.56
  29 px * 2.56 = 74 px                                trivial

The interesting row is the last one. High detail on the full page is marginal for 10-point body text — right in the band where the model will read most of it and quietly invent the rest. Cropping the region first is not a cost optimisation here; it is the difference between a task that works and one that fails intermittently, which is the worst failure mode there is.

Running the A/B properly

Three things make the difference between a useful comparison and a number you will not trust in a month.

• Fix everything but the flag. Same images, same prompt, temperature 0, same model version pinned. Detail level and prompt wording interact, so changing both at once tells you nothing.
• Stratify the sample. Sixty images split into the categories you actually receive — clean digital PDF, phone photo at an angle, fax-quality scan — beats 500 images that are all the easy case. The aggregate number will be dominated by whichever category is most common, which is not what you need to know.
• Score abstention separately from error. “I cannot read that field” is a good outcome and a wrong value silently returned is a bad one; a single accuracy percentage merges them. Give the model an explicit way to abstain in the schema, then count the three outcomes.

Compute the cost side from the response usage counts rather than the formula, and you end the afternoon with cost per correct extraction for each setting, which is the only figure that decides anything.

Routing per image, not per app

The setting does not have to be global. Most pipelines have a cheap local signal for which branch an image needs — often just its dimensions, sometimes an edge-density or text-density estimate from OpenCV, sometimes the document type you already know from the upload form. Route on that: thumbnails and product photos to low, dense documents and screenshots to high, and anything the low pass abstained on to a high-detail retry. That last rule is a cascade, and it usually costs less than running everything at high detail while producing the same answers.

There is one more setting hiding behind the flag, and on many stacks it is the better lever: sending several targeted crops instead of one image. A form with four fields of interest can be four small crops at native resolution, each of which is cheap and each of which is perfectly legible, rather than one expensive full page in which all four are marginal. It costs more requests and less money, and it moves the accuracy question from “can the model read this” to “do I know where to look”, which for a known document layout is a solved problem you can answer with template coordinates rather than a model call.

Whatever you settle on, pin it. Detail defaults have changed between model versions before, and a pipeline that relied on an implicit default can start costing several times more, or start failing on small text, without a single line of your code changing. Set the parameter explicitly on every request even when your choice matches today’s default, and assert on the image token count in the response so a silent change shows up as a failing check rather than as a bill.