Part-of-Speech Tagging and Dependency Parsing

Part-of-speech tagging is the most solved problem in NLP that people still argue about. English accuracy has sat around 97% for two decades. The question is not whether it works — it is whether the structure it produces is worth anything now that a model can read the sentence directly.

What a tagger gives you

A tagger assigns each token a grammatical category. Two tagsets dominate. The Penn Treebank set (Marcus, Santorini and Marcinkiewicz, 1993) has around 45 tags and encodes fine distinctions — NN singular noun, NNS plural, NNP proper singular, VBD past tense verb, VBG gerund. Universal Dependencies uses seventeen coarse tags (NOUN, VERB, ADJ, PROPN, …) designed to be comparable across a hundred-plus languages, with the fine detail moved into separate morphological features.

The hard part is ambiguity, and it is more common than intuition suggests. book is a noun or a verb; that is a determiner, pronoun, or complementiser; -ing forms are verbs, adjectives or nouns depending on context. Resolving these requires looking at neighbouring words, which is why taggers were sequence models from the beginning — hidden Markov models, then maximum-entropy, then neural.

The 97% ceiling, and what is left

Toutanova, Klein, Manning and Singer’s cyclic dependency network tagger (2003) reported about 97% token accuracy on the Penn Treebank Wall Street Journal, and that number has moved remarkably little since despite everything that has happened to the field. Manning wrote a paper about precisely this — Part-of-Speech Tagging from 97% to 100%: Is It Time for Some Linguistics? (CICLing, 2011) — and his analysis of the residual errors is the useful part: a substantial share of them are inconsistencies or genuinely debatable cases in the gold annotation itself, not model failures.

Two consequences. First, treat 97% as effectively the measurement ceiling for English newswire; a paper claiming a large jump is probably measuring something else. Second, do the arithmetic on what per-token accuracy means per sentence: at 97% per token, a 20-token sentence has roughly a 0.97²⁰ ≈ 54% chance of being entirely correct. If your rule depends on every tag in the sentence being right, it is wrong about half the time, and that is with a state-of-the-art tagger on the domain it was trained on.

Dependency parsing in one picture

A dependency parse adds the edges: each word points to its syntactic head with a labelled relation. In the bank approved my loan, approved is the root, bank is its nsubj, and loan is its obj, with my hanging off loan as a possessive. The result is a tree over the tokens rather than a bracketed constituent structure, which is why it became the standard — it is easier to produce, easier to evaluate, and translates better across languages with freer word order.

What you can do with the tree is answer structural questions in code rather than in a prompt: who is the subject of this verb, which adjectives modify this noun, is this clause negated, what is the head of this noun phrase. Those queries are exact, instant and free.

“Free” is close to literal here, and it is the reason this page exists in a cluster about when to pay for a model. A small statistical tagger and parser run on a CPU, in-process, with no network call, on a model of a few tens of megabytes. Universal Dependencies publishes treebanks for well over a hundred languages, and toolkits such as spaCy and Stanza (Qi, Zhang, Zhang, Bolton and Manning, ACL 2020 system demonstrations) ship pre-trained pipelines built from them. The marginal cost of tagging one more document is the CPU time and nothing else, which means you can afford to run it over an entire corpus as a preprocessing pass — something you would never do with a per-document API fee attached.

The catch is the same one that applies to every supervised component here: those pipelines were fitted on the treebank’s domain, which is usually edited prose. Tagging accuracy on chat messages, product listings, OCR output or clinical shorthand is materially lower, and the published newswire figure tells you nothing about it. If a rule you are writing depends on the tags, sample fifty sentences of your own text and check them by hand before trusting the rule.

import spacy
nlp = spacy.load("en_core_web_sm")
doc = nlp("The battery drains fast but the screen is not bright enough.")

# noun phrases, as candidate aspects to attach sentiment to
print([chunk.text for chunk in doc.noun_chunks])
# -> ['The battery', 'the screen']

# every adjective, with the noun it actually modifies
for tok in doc:
    if tok.pos_ == "ADJ":
        negated = any(c.dep_ == "neg" for c in tok.children)
        print(tok.text, "->", tok.head.text, "negated" if negated else "")

Four places syntax still earns its keep

• Candidate generation for keyphrases. The most reliable keyphrase filter is a POS pattern — adjective* noun+ — and it costs nothing. See the extraction methods, where every graph-based method depends on this filter to avoid proposing verbs and function words.
• Aspect extraction in reviews. Noun chunks give you the things being talked about; the dependency edges tell you which adjective belongs to which. That is how you get battery: negative, screen: negative out of the sentence in the code above rather than one overall score — the distinction aspect-based sentiment is about.
• Negation scope. Finding a neg edge and walking its subtree is an exact answer to “what is being denied”. Keyword matching cannot do this, and it is the single most common cause of a rule-based classifier getting a document exactly backwards.
• Correct lemmatisation. A lemmatiser needs a part of speech to distinguish the noun meeting from the verb meet. The tagger is what supplies it.

When to skip it entirely

Do not build a syntax stage in order to understand a sentence. That was the 1990s programme and it was superseded for a reason: a pipeline of tagger, parser and hand-written tree rules compounds errors, is brittle across domains, and needs a specialist to maintain, while a model reads the sentence in one step. If what you want is the meaning, ask for the meaning.

The remaining value of syntax is as a cheap, deterministic structural filter that runs on every document before anything expensive does — narrowing candidates, supplying offsets, answering structural questions exactly. Used that way it costs microseconds and never surprises you. Used as a substitute for comprehension it will disappoint you at exactly the rate the per-sentence arithmetic above predicts.