Measuring answer quality

The one thing we cannot measure

Cost and latency we measure ourselves, per request, exactly. Whether the answer was any good is the number that actually decides which model to keep paying for, and only you ever find out — usually when a user clicks thumbs-down. This is where you send it back.

Sending a score

curl https://api.multigrid.ai/v1/feedback \
  -H "Authorization: Bearer $MULTIGRID_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "request_id": "req_…",
    "score": -1,
    "comment": "Invented a refund policy we do not have."
  }'

It is idempotent per request id: a second score replaces the first, because a corrected rating is a correction and not a second opinion. An id that is not yours reads as a 404 — otherwise anyone holding any valid key could probe for the existence of other people’s requests.

Using the scores

The point is comparison against the numbers you already have. A model that is 40% cheaper and scores the same on your own traffic is a decision; without the score it is a guess. Pair feedback with GET /usage’s per-model breakdown, and use metadata labels so you can slice by the feature the score came from rather than by model alone.

For a controlled comparison rather than an observational one, provider.weights will split live traffic between two providers, and a pinned preset version keeps a configuration frozen for as long as you are measuring it.

What is not built