Generative Answers vs Ten Blue Links

A ranked list and a generated answer are usually discussed as two designs for the same feature. They are not. One returns candidates and delegates the judgement; the other makes the judgement and hides the candidates, and everything that follows — the cost, the failure modes, the evaluation — comes from that.

They are different products

The ten-link format has a property that is easy to overlook because it is so familiar: it lets the user triangulate. Three snippets that agree are evidence; a fourth that disagrees is a signal to look closer. The user is doing synthesis, and the interface is honest about the fact that synthesis is happening.

A generated answer does the synthesis and presents the result as a single statement. When it is right, that is a large saving of the user’s time. When it is wrong, the user has no way to notice, because the disagreement that would have been visible across snippets has been resolved inside the model and thrown away. That asymmetry is the whole argument, and it is why the format choice depends far more on the cost of a wrong answer than on anything about the model.

Cost per query, derived

Every number below is an assumption. Substitute yours; the structure is what survives a price change.

ASSUMPTIONS
  retrieved passages        8
  tokens per passage        400
  system prompt + query     300 tokens
  answer length             300 tokens
  price, input              $0.50 per 1M tokens
  price, output             $1.50 per 1M tokens

INPUT   8 * 400 + 300 = 3,500 tokens
OUTPUT                 =   300 tokens

cost = 3,500 * 0.5e-6  = $0.001750
     +   300 * 1.5e-6  = $0.000450
                       = $0.002200 per query

at 1,000,000 queries / day  ->  $2,200 / day  ->  ~$803,000 / year

Against that, a retrieval-only query costs a fraction of a cent in compute and no per-token fee at all. The ratio is two to three orders of magnitude, and it does not shrink with scale the way infrastructure costs do, because it is a marginal cost per query rather than a fixed one.

Three levers move the number and they are worth knowing in order of size. Generating answers for only a fraction of queries — the ones an intent classifier says are answerable — cuts cost linearly in that fraction, and it is by far the largest lever. Caching answers for repeated queries exploits the same heavy head that makes query rewriting cacheable. And retrieving fewer or shorter passages attacks the dominant term in the input count, since 3,200 of the 3,500 input tokens are passages. Latency follows the same ordering: not generating is instant, a cache hit is instant, and a shorter context has a shorter prefill.

A decision rule by intent

The format should be chosen per query, by the intent classifier described in query understanding, rather than globally by a product decision. What each intent wants:

Notice that the table almost never says “answer instead of links”. The format that wins most often is an answer above the links rather than in place of them, and the reason is the asymmetry from the first section: the links are what let a reader check the answer, and a reader who can check is a reader who can forgive a mistake. Replacing the list is a much stronger claim about your retrieval quality than adding to it, and it should be made deliberately rather than because the answer looked good in a demo.

The other structural decision is who chooses. A per-query classifier deciding the format is one option; letting the user decide, with a preference that persists, is another and is considerably easier to get right. Both beat a global setting, because the same person wants different formats on different days for different questions.

The failures that are specific to answers

• Attribution drift. The answer carries a citation, the citation is a real document, and the sentence it is attached to is not supported by that document. This is the most damaging failure in the format because the citation is what the user is trusting. Span-level attribution — the check that the cited passage actually entails the sentence — is the fix, and it is worked through in citations and source attribution.
• Silent merging. Two retrieved passages describe different things — different product versions, different jurisdictions, different years — and the answer merges them into one statement that is true of neither. A list of links would have shown the user two dates; the answer shows one.
• Retrieval failure hidden by fluent prose. When retrieval returns nothing useful, a ranked list looks empty and a generated answer looks confident. The mitigation is a relevance threshold below which you refuse to generate and fall back to links, and it is the single highest-value guardrail in the format.
• Evaluation that does not transfer. nDCG measures the ordering of a list and says nothing about a paragraph. Answer quality needs its own evaluation — faithfulness to the retrieved context, coverage of the question, and correct refusal — which is a different harness from the one in the metrics page. Teams that ship generation on top of a search stack usually keep measuring the search stack and stop measuring anything that matters.