Batch jobs

Queue up to 50,000 requests from one JSONL file and collect them later.

5 min read

The file is the body

This is not OpenAI's batch API

client.batches.create(input_file_id=…) cannot work here and will fail confusingly if you try it: it sends a small JSON object, this endpoint reads the body as a JSONL file, and you get a 400 listing parse errors for a body that was never JSONL. There is no /v1/files and no input_file_id. Cancel is DELETE /batches/{id}, not POST /batches/{id}/cancel. The response carries cost_ceiling_usd and results_url rather than output_file_id and expires_at.

Everything else on this gateway is OpenAI-shaped on purpose, so it is worth being blunt about the one surface that is not. Use curl --data-binary, or any HTTP client — there is no SDK method to reach for and pretending otherwise wastes an afternoon.

No multipart, no upload step, no file id to keep track of. POST the JSONL itself; curl --data-binary @requests.jsonl is the whole client.

requests.jsonl

{"custom_id": "ticket-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "openai/gpt-5-nano", "messages": [{"role": "user", "content": "…"}]}}
{"custom_id": "ticket-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "openai/gpt-5-nano", "messages": [{"role": "user", "content": "…"}]}}

bash

curl -X POST "https://api.multigrid.ai/v1/batches?name=nightly-triage&completion_window_hours=24" \
  -H "Authorization: Bearer $MULTIGRID_API_KEY" \
  -H "Content-Type: application/jsonl" \
  --data-binary @requests.jsonl

Each line needs custom_id, method, url and body. url is either /v1/chat/completions or /v1/embeddings, and every line in one file must use the same one. The body is exactly what you would have posted to that endpoint — including models and provider if you want them.

Validation happens up front and reports up to 25 problems at once, each with its line number and what to put instead. A file with one bad line is rejected whole rather than half-run. Those problems arrive in an issues array beside the usual error object — the one place in the API where the error envelope grows a field rather than only changing its contents — plus issues_omitted when there were more than 25.

Limits

Limit	Description
50,000 lines	Per file. The whole file is parsed in memory and every line becomes a row.
16 MB	Per file. Split larger jobs.
128 characters	Maximum `custom_id` length. It is how you match results back to inputs, so it must be yours and unique.
1 to 168 hours	The completion window, defaulting to 24. Passed as `?completion_window_hours=`.

Submitting and collecting

POST /batches — submit. The response includes the parsed line count and a cost ceiling. ?name= labels the job so it is identifiable in the dashboard later, and X-Multigrid-Batch-Name does the same thing if a header suits your client better.
GET /batches — every batch on the account, newest first. Capped at 100 with no cursor: an account with more than that cannot currently page through them all from the API, and the dashboard is the answer until it can.
GET /batches/{id} — progress and counts for one.
DELETE /batches/{id} — cancel. Stops dispatch; items already upstream are left to finish and are billed honestly, and everything that ran stays downloadable.
GET /batches/{id}/results — the results, keyed by your custom_id, in submitted order so the output zips against the input without a lookup table. ?filter=failed narrows it to the items that did not run, which is the file you actually want when you are about to resubmit; ?filter=succeeded does the opposite. The response carries X-Multigrid-Batch-Status, so a partial file is explicable rather than alarming — results are downloadable while the job is still running, because a long batch is a long time to wait to find out whether the prompt is even right.

A batch needs an inference key, and its management calls count against that key’s ordinary per-minute rate limit — a polling loop that did not would be the cheapest possible way to ignore the limit. Every response carries X-RateLimit-Remaining, so a poll loop can pace itself instead of finding out with a 429.

What it costs

Batching is not cheaper here

Other gateways sell batch at a discount because they resell tokens at a margin and can give part of it back. Multigrid passes provider list price straight through, so there is no margin to discount and a batched token costs exactly what a live one costs. Batch is for throughput and for not writing your own queue — not for the bill.

The quote you get at submission is a ceiling, not a forecast: it assumes every prompt is as long as it looks and every completion runs to its full max_tokens. Real batches land well under it. It is deliberately the same pessimistic arithmetic the live gateway uses for its balance check, so a batch cannot be admitted that the gateway then refuses item by item.

Something here disagrees with what the API actually did? That is a bug in this page, and worth reporting.

Report it