Keyword Extraction and Summarising Without Generation

Extraction picks spans out of the source. Generation writes new text. That distinction is the whole decision: extraction cannot invent anything, and generation cannot be prevented from it. Everything else — cost, speed, quality — follows from which guarantee you need.

Three tasks people conflate

• Keywords. Single terms that characterise a document. Useful for faceting, tagging and index expansion.
• Keyphrases. Multi-word units — battery life, data processing agreement. Almost always what people actually want, and harder because you must decide where a phrase begins and ends.
• Extractive summaries. A selection of whole sentences from the document, in document order. Not a summary in the human sense — nothing is condensed or rephrased — but every word is verifiably from the source.

Statistical methods

TF-IDF top-n. Score every term with TF-IDF, take the highest. It requires a corpus to compute document frequencies against, which is both its strength — the weighting is specific to your collection — and its limitation, since you cannot run it on a single document in isolation.

RAKE (Rose, Engel, Cramer and Cowley, 2010) needs no corpus at all. Split the text at stop words and punctuation; the fragments between them are candidate phrases. Score each word by the ratio of its degree — how many other words it co-occurs with in candidates — to its frequency, and sum across the phrase. It is a few dozen lines, runs on one document, and is remarkably decent on technical prose. This is also the one place a stop word list is doing real structural work rather than being copied out of a tutorial.

YAKE (Campos, Mangaravite, Pasquali, Jorge, Nunes and Jatowt, Information Sciences, 2020) is the more considered single-document method: it combines casing, position in the document, term frequency, how many different sentences a term appears in, and its dispersion in context into one score. Unsupervised, language-agnostic, no training, no corpus.

Graph methods

TextRank (Mihalcea and Tarau, EMNLP 2004) applies PageRank to text. Build a graph whose nodes are candidate words — filtered by part of speech, typically nouns and adjectives, which is where a tagger earns its keep — with edges between words appearing within a small window of each other. Run PageRank. The highest-ranked words are keywords, and adjacent ones in the original text are merged into keyphrases.

The intuition transfers directly from the web: a word is important if it co-occurs with other important words. LexRank (Erkan and Radev, JAIR 2004) is the same idea over sentences, with edges weighted by cosine similarity, and it produces extractive summaries rather than keywords.

KeyBERT is the modern variant: embed the document, embed each candidate phrase, and rank candidates by cosine similarity to the document embedding, with a diversity penalty so you do not get five paraphrases of the same idea. It captures semantic centrality that co-occurrence misses, at the cost of an embedding model.

Extractive summaries and the lead baseline

Before building anything here, know the baseline you have to beat. On news summarisation, simply taking the first three sentences of the article — the “lead-3” baseline — is famously competitive with elaborate systems, a point See, Liu and Manning noted in their pointer-generator work (ACL 2017). That is not a fact about summarisation; it is a fact about news, where journalists are trained to front-load. On a corpus with a different structure — transcripts, forum threads, contracts — the lead baseline is useless, and a centrality-based extractive method has something to add.

The general lesson is worth more than the technique: compute the trivial baseline on your own documents first. If lead-3 gets you 80% of the way, the interesting question is whether the remaining 20% justifies any of this.

When generation is worth paying for

First, the cost correction, because “extraction is free” is not true. Assumptions, replaceable: 1 million documents, an assumed 5 ms per document for YAKE or TextRank on one core, and $0.05 per hour for a 4-vCPU instance. That is 5,000 core-seconds, about 1.4 core-hours, roughly $0.02 — plus the engineer who wrote it and the box that was already running. The honest claim is not that it costs nothing; it is that there is no per-document API fee, so the marginal cost of the ten-millionth document is the same as the first.

Compare: 1 million documents at 300 tokens is 300 million input tokens, and a summary of 60 output tokens each is 60 million output tokens. Output is the expensive meter — see why output costs several times input — so at an assumed $0.10 input and $0.40 output per million, that is $30 + $24 = $54 per million documents, recurring. Roughly three orders of magnitude apart, again.

And yet generation is genuinely better at three things extraction cannot do at all. It can fuse information stated in three separate places into one sentence. It can abstract — say “several customers reported billing problems” when the source lists eleven individual complaints. And it can adapt register, turning a legal paragraph into a sentence a customer understands. No extractive method will ever do any of these, because they all require writing words that are not in the document.

Which is exactly the risk, stated from the other side: a generated summary can assert something the source does not say, and that failure is invisible without checking — the subject of evaluating summaries. So the decision is not about quality. If the output is faceting, tagging, search snippets, or anything at a volume where a per-document fee compounds, extract. If the output is read by a person who will act on it, and being wrong matters more than being cheap, generate — and budget for verifying it.