Building a Multimodal Pipeline That Doesn't Cost a Fortune

Multimodal bills grow faster than text bills because a single image can cost as much as a thousand words and nobody notices until the invoice arrives. Three levers move the number materially. All three are arithmetic, and the arithmetic is on your numbers rather than anyone else’s.

Write the cost model first

Before optimising anything, write the equation down. It takes ten minutes and it usually reveals that the term you were about to attack is not the large one.

monthly = N * (img_tokens * in_rate
               + prompt_tokens * in_rate
               + out_tokens * out_rate)

worked example, placeholder rates -- substitute yours:
  N               = 100,000 images / month
  img_tokens      = 1,400        (high detail, one page)
  prompt_tokens   = 600          (system prompt + schema)
  out_tokens      = 400
  in_rate         = $1.00 / M tokens
  out_rate        = $4.00 / M tokens

  images   100,000 * 1,400 / 1e6 * 1.00 = $140.00
  prompt   100,000 *   600 / 1e6 * 1.00 =  $60.00
  output   100,000 *   400 / 1e6 * 4.00 = $160.00
                                          -------
                                          $360.00

Note what that shows. The images are 39 % of the bill and the output is 44 %. If you had spent the week on image compression you would have optimised the second-largest term while a verbose response schema quietly cost more. The system prompt is a sixth of the total and is identical on every call, which makes it the most cacheable thing in the equation.

Lever 1: downscale

Because token count follows image area, the saving is quadratic in the linear scale factor. Halving both dimensions quarters the image tokens.

2048 x 1536 -> area 3.1 MP
1024 x  768 -> area 0.79 MP     = 25 % of the tokens
 768 x  576 -> area 0.44 MP     = 14 % of the tokens

on the model above, the image term:
  $140.00 -> $35.00 at half linear size

The constraint is the minimum feature size — see the detail-levels page for the division that tells you how far you can go before the thing you need to read stops being legible. Two practical notes: strip EXIF and re-encode as a reasonable-quality JPEG for upload (this affects transfer time and storage, not tokens), and crop before you scale. Cropping to the region of interest reduces area without reducing the resolution of what remains, which is strictly better than scaling when you know where to look.

Lever 2: cache, at three levels

• Result cache. Key on a hash of the image bytes plus the prompt version. If the same image is ever processed twice — user retries, re-runs, duplicate uploads, a batch job re-run after a failure — this is a 100 % saving on the repeats. In pipelines with any human in the loop the duplicate rate is routinely a double-digit percentage, and almost nobody measures it before deciding this is not worth building.
• Prompt caching. Where the provider supports it, a long shared prefix is billed at a reduced rate. In the worked model above the 600-token system prompt is identical on every call. Ordering matters: the cacheable prefix must come first, so put the system prompt and schema before the image, not after.
• Derived-artefact cache. Cache what you extracted, not just the answer. If you have a transcript of a video or an OCR pass over a page, every subsequent question can be answered from text at a fraction of the cost of re-sending the pixels. This is the largest of the three for anything a user asks follow-up questions about.

Lever 3: cascade, and its break-even

Run a cheap model first, escalate only what it cannot handle. The question is always whether the escalation rate is low enough for the extra pass to pay for itself, and that is one line of algebra:

cascade   = N * c_cheap + N * e * c_expensive
baseline  = N * c_expensive

worth it when   c_cheap + e * c_expensive  <  c_expensive
i.e.            e  <  1 - (c_cheap / c_expensive)

c_cheap = $0.001, c_expensive = $0.010:
  break-even escalation rate e < 1 - 0.1 = 90 %

c_cheap = $0.005, c_expensive = $0.010:
  break-even e < 50 %

at c_cheap = $0.001, c_expensive = $0.010, e = 20 %:
  cascade  = 100,000 * 0.001 + 20,000 * 0.010 = $300
  baseline = 100,000 * 0.010                  = $1,000
  saving 70 %

The break-even is far more permissive than people expect: with a ten-times price gap, a cascade wins until nine out of ten requests escalate. What breaks a cascade is not the escalation rate but bad escalation decisions — if the cheap model cannot tell when it is wrong, you either escalate everything (no saving) or accept its errors (no quality). So the routing signal is the whole design:

• Ask for abstention explicitly. A required “not determinable” option in the schema gives you a clean escalation trigger at no cost.
• Validate structurally. Fails schema, fails a checksum, line items do not sum to the total — escalate. These are free and catch a large share of real errors.
• Route before you call. Image dimensions, file type, document class from the upload form. Cheapest possible signal, and frequently good enough.
• Measure escalation rate as a first-class metric. It drifts as your input mix changes, and the whole economic case rests on it.

The order to apply them

Instrument first: log image tokens, output tokens and cost per request from the response usage object, not from your own arithmetic. Then, in this order, because each step is cheaper to implement than the last and they compound: tighten the output schema, since output tokens are usually the dearest per unit and are entirely under your control; downscale and crop; add the result cache; enable prompt caching and reorder the prompt so the prefix is stable; and only then build the cascade, which is the most engineering for the least certain payoff until the others are done.

One last piece of arithmetic worth doing before any of it: multiply your current per-request cost by your projected volume in twelve months. Multimodal pipelines are almost always prototyped at a volume where nothing matters and deployed at one where everything does, and the decisions that are cheap to make now — schema shape, cacheable prompt ordering, storing derived artefacts — are expensive to retrofit.

One lever sits outside this list because it costs nothing to implement and does not apply to every workload: batch processing. Where a provider offers an asynchronous batch endpoint with a turnaround measured in hours, it is typically priced at a substantial discount to the synchronous rate. Any multimodal pipeline whose work is not user-facing — overnight document ingestion, back-catalogue tagging, re-processing after a prompt change — is a candidate, and the change is usually a different endpoint rather than a different architecture. Sort your traffic into what a person is waiting for and what nobody is waiting for before optimising anything else; the second pile is often larger than expected and is the cheapest thing on this page to move.