E-commerce Search: Attributes, Stock and Intent

Product search inherits the machinery of document search and almost none of its assumptions. The documents are twelve words long, the objective is not relevance, and the most similar item is sometimes the worst possible result.

Four things that are simply different

• There is barely any text. A product is a title, a few attributes and a description written by a supplier. BM25’s term-frequency saturation has nothing to saturate: every term appears once. Almost all of the signal has to come from attributes, behavioural data and the query parse rather than from the text scoring that dominates document search.
• Relevance is necessary and not sufficient. An out-of-stock item in the wrong size at the wrong price can be a perfect textual match and a useless result.
• The catalogue moves constantly. Prices, stock and promotions change hourly. A ranking that reads any of them has an index freshness requirement measured in minutes, which is a different engineering problem from a document index rebuilt nightly.
• The labels are free and biased. Purchases are a strong relevance signal and you get them without paying annotators. They are also produced by whatever your ranker showed, which is the position-bias problem from the metrics page with money attached.

The objective is expected value

Document search ranks by relevance because relevance is the goal. A shop is trying to do something else, and it is worth writing that down honestly:

score(i | q) = P(purchase | q, i) * value(i)

  P(purchase | q, i)  probability this shopper buys this item
  value(i)            margin, or lifetime value, or whatever the
                      business is actually optimising

Two products, with every number an assumption:

item A   P(purchase) = 0.02   margin = $12   ->  EV = $0.24
item B   P(purchase) = 0.05   margin =  $4   ->  EV = $0.20

ranking by conversion:      B, A
ranking by expected value:  A, B

The two orderings disagree, and which one is right is a business decision rather than a search one. But there are two constraints on it that a search team should insist on, because they are the difference between a ranking and a con.

First, the value term must be bounded relative to the relevance term. If margin can promote an item past a materially better match, the shopper eventually notices, and the cost of that shows up as a slow decline in search usage that no A/B test running for two weeks will catch. Capping the value term, or using it only to break ties between items of comparable predicted relevance, keeps it honest. Second, the effect has to be measured on a horizon long enough to include returns and repeat purchases. Margin-weighted ranking reliably improves the metric it is optimising over a two-week window; whether it improved anything is a different question with a longer answer.

The query is a structured constraint

“red running shoes size 10 under 80” is not text to be matched. It is a parse:

colour     = red
category   = running-shoes
size       = 10          (in which sizing system?)
price_max  = 80          (in which currency?)

Once that parse exists the retrieval problem is nearly trivial, which is why attribute extraction is where the effort belongs. Three rules that matter more than the extraction model:

• Hard constraints and soft preferences are different. Size is a hard constraint — a shoe in the wrong size is not a worse result, it is a wrong one. Colour usually is not: a shopper who asked for red will look at the burgundy one. Extracting both into filters is the most common cause of an empty results page, and the multiplicative arithmetic for that is in faceted search.
• Units need a locale. Size 10 is three different shoes depending on the country, and a price ceiling is meaningless without a currency. An extractor that produces a number without a unit has extracted nothing.
• Extraction confidence should drive the boost. A confident parse becomes a filter, an unconfident one becomes a ranking boost. Treating every extraction as certain is how a query mentioning a colour in passing ends up filtering the catalogue down to nothing.

Stock deserves its own treatment because it is a hard constraint that changes minute to minute. The standard design is variant-level: the product stays findable if any variant is purchasable, and out-of-stock variants are visible but not offered. Removing the product entirely means a shopper who searched for it by name concludes you never carried it, which is worse than showing them a restock notification.

Near-miss substitution

This is the failure that makes product search genuinely different, and it is the one that embeddings make worse rather than better.

A shopper searches for a case for a specific phone model. The catalogue contains cases for the previous model. Their titles differ by a single token, so by any text similarity measure — lexical or dense — they are nearly identical, and a ranker optimising similarity will happily put them on the page. The result is not slightly wrong. It is an item that physically does not fit, bought, shipped and returned, and the cost is the return plus the trust.

The rule that follows is worth stating flatly: compatibility tokens are constraints, not similarity signals. Model numbers, generations, part numbers, sizes, voltages, thread pitches. When one appears in the query, it becomes an exact-match requirement that no amount of semantic similarity can override. Implement it as a filter on an extracted attribute, not as a boost, and test it explicitly — a fixture of queries containing model numbers, asserting that nothing from an incompatible generation appears, is twenty lines and catches the regression every time somebody retunes the embedding weight.

The general form of this is that dense retrieval is bad at exactly the queries where a small string difference means a large real-world difference. That is the same weakness described in semantic versus keyword search, and in a catalogue it has a price tag.

Labels, and a public dataset

Binary relevance is too coarse for shopping, because the interesting cases are in the middle. The label scheme worth adopting is the one used by the Shopping Queries Data Set released by Reddy and colleagues (2022), a public multilingual dataset of query-product pairs judged on four levels:

The value of the four-way scheme is that it lets you say something a binary label cannot: substitutes are welcome below the exact matches and harmful above them. That is a graded judgement, which is exactly what nDCG consumes, and it makes the offline metric measure something a shopper would recognise. Being a public dataset it is also the cheapest way to sanity-check a retrieval stack before you have judgements of your own.