Why Short Queries Embed Worse in Morphologically Rich Languages

Long Russian documents retrieve fine. Two-word Russian queries do not, and the difference is arithmetic: inflection changes a fixed number of tokens, and a fixed number of tokens is a rounding error in a passage and most of a query.

A two-word query in six forms

Russian nouns inflect for six cases, and adjectives agree with them. A user searching for a red book will type one of these, depending on how they mentally complete the sentence:

красная книга    nominative   "a red book"        (the dictionary form)
красной книги    genitive     "of a red book"
красной книге    dative       "to a red book"
красную книгу    accusative   "a red book" (object)
красной книгой   instrumental "with a red book"
красной книге    prepositional "about a red book"

Both words change. Every time.

The English comparison is the point. “red book”, “of the red book”, “with a red book” — the two content words are byte-identical in all of them, and the case is carried by separate function words that are among the highest-frequency tokens in the model’s vocabulary and contribute little to the pooled representation. English encodes the grammatical relation in tokens that barely matter; Russian encodes it inside the tokens that matter most.

Now put the document in a different case from the query, which is the normal situation — the document says в красной книге because it is describing what is in the book, and the searcher types the dictionary form. Every content word differs in surface form between the query and the passage that answers it.

The same change, two different proportions

Here is the whole mechanism in one comparison. Sentence embeddings pool over tokens, so the effect of changing some tokens is roughly proportional to the share of the pool they represent.

Query: "красная книга"        ~5 tokens, 2 of them inflectional endings
       inflection changes         ~40% of the pooled inputs

Chunk: 512-token passage that also inflects the same two words
       inflection changes         ~0.4% of the pooled inputs

A long passage is robust because it has hundreds of other tokens agreeing with each other about what it is about, and because it almost certainly contains the word in several cases anyway. A two-word query has no such redundancy. It has five tokens, two of which just changed, and nothing else to anchor the meaning.

This is why the failure looks like a language problem and is really a length problem that the language makes worse. The same model on the same Russian corpus with ten-word natural-language queries behaves far better, because the ten-word query has enough unchanged material to survive the endings. If your search box produces two-word queries, that is the thing to design around.

Subword stem sharing is luck, not design

The obvious objection is that a subword tokenizer should handle this automatically: книга and книге both start with the stem книг, so surely they share a token and the endings are just a suffix each.

Sometimes they do, and when they do the model handles the pair well. But that outcome is not guaranteed by anything. BPE merges are applied greedily over the byte sequence, and which merges apply depends on the whole string, so a frequent inflected form may be absorbed into a single token while a rarer form of the same word is carved differently and does not preserve the stem boundary at all. Cyrillic characters are two bytes in UTF-8, so merges can also land mid-character in the byte stream for rare sequences, which produces fragments that correspond to nothing linguistic.

So stem sharing is a frequency effect. Common words in common cases get it; the rest do not, and “the rest” includes most domain vocabulary, which is exactly what people search for. The way to find out is to print the token ids for the six forms of five words from your own domain and look at how many units they share. It takes two minutes and it converts an argument into an observation.

The same reasoning transfers directly to agglutinative languages and is worse there. Finnish has fifteen cases and stacks possessive and clitic suffixes on top; Turkish and Hungarian build long words from strings of suffixes, with Turkish vowel harmony changing the suffix vowels to match the stem, so the same grammatical suffix has several surface forms. A single Turkish word can correspond to an English clause, which means the “short query” problem starts at one word.

Agreement multiplies the problem

Case marking alone would be manageable if only nouns changed. Agreement means the change propagates. In the Russian example, the adjective changes because the noun did. Add a number and it changes too. Add a past tense verb and it agrees in gender with its subject. A short phrase that would have one variable ending in a language with agreement only on the head can have three or four.

There is a related trap in how these queries reach the index. Many embedding models are trained asymmetrically, with an instruction or prefix distinguishing a query from a passage — some expect a literal query: and passage: prefix, others a longer instruction sentence. Those prefixes exist precisely because short queries and long passages have different statistics, and omitting them costs the most on the shortest queries, which is this case. Check your model’s card and apply the prefix on both sides.

Fixes, in order of effort

1. Apply the model’s query prefix. Free, and it is the fix most often missing. Verify the exact string from the model card rather than guessing.
2. Add a lemmatised lexical field. Run a morphological analyser over documents and queries — pymorphy or a Snowball stemmer for Russian, and language-specific analysers elsewhere — and index the lemmas for BM25. Lemmatisation is the direct answer to inflection, and it solves it completely for the lexical path. Fuse the lexical and vector results by rank.
3. Expand the query. Generate the two or three most likely case forms and issue them as separate queries, merging results. For a two-word query this is cheap and the recall gain is the largest single lever after lemmatisation.
4. Lengthen the query. If your interface can, turn the keyword query into a sentence before embedding — a small model rewriting two words into a natural question adds anchoring tokens and dilutes the endings. This costs a call per query and helps disproportionately at very short lengths.
5. Rerank. A cross-encoder reads query and candidate together and is far less sensitive to surface form than two independent vectors are. Over-retrieve, then rerank the top fifty.