AI in E-commerce: Search, Descriptions, Recommendations

On-site search converts far better than any other entry point on a store, and when it is bad the ranker gets rebuilt. The ranker is usually not the problem. The catalogue is missing the attribute the shopper searched for, and no amount of semantic retrieval invents a fact that is not in the record.

Search fails on the catalogue, not the ranker

A shopper searches for a waterproof jacket with pit zips in a size 16. The store has eleven of them. Nine have no waterproof_rating, none has a ventilation attribute at all because the field was added last year and never backfilled, and size is a variant property spelled three ways across suppliers. Every one of those failures is a data completeness failure, and every one of them presents to the merchandising team as “search is broken”.

This is the highest-value thing a language model does in e-commerce and it is nearly invisible: reading the supplier’s spec sheet, the manufacturer’s PDF, the existing long description and the images, and emitting the structured attributes the catalogue schema declares. It is an extraction task with a fixed schema, a closed vocabulary per field, and — crucially — a ground truth you can build, because a merchandiser can label two hundred products in an afternoon.

Evaluate it per attribute, not overall. Colour will be near perfect; material will be good; care instructions will be fine; anything involving a measurement in the wrong unit system will be poor. Reporting a single figure hides that heel_height is the field that will embarrass you. And build an abstain path per field: an empty attribute is a missing filter, but a wrong attribute is a return, and returns cost shipping both ways plus the item’s remaining sellable life. See structured extraction and enums versus free text, which is exactly the choice a catalogue schema forces.

What an e-commerce query actually looks like

Product search queries are unlike the queries embedding models are usually demonstrated on. They are two or three words, heavily misspelt, and a large share of them are identifiers: a part number, a model code, an ISBN, “RN4128-B”. Dense retrieval is actively bad at those, because an embedding of an alphanumeric code is not meaningfully close to the embedding of the same code with one character different, and a nearest-neighbour search will happily return a similar-looking but different part. Lexical matching with a good analyser wins outright on that class.

Meanwhile dense retrieval wins on the descriptive queries lexical search fails at — “something warm for a toddler that isn’t scratchy”. So the answer is hybrid, and the interesting engineering is in the fusion and in knowing which class a query is. The mechanics are in hybrid search and semantic versus keyword search.

The operational metric to watch is the null-result rate — the share of searches returning nothing — segmented into three buckets, because the fixes are unrelated: queries for products you do not stock (a merchandising signal, and the single most valuable free market research a store has), queries for products you do stock but cannot find (a retrieval or attribute problem), and misspellings (a normalisation problem). Storefronts routinely report one number and then argue about which cause it is.

Descriptions: the two costs nobody budgets

Generating product copy across a catalogue is the most commonly attempted project here and it has two costs that do not appear in the token estimate.

The first is near-duplication. One prompt over a catalogue of forty thousand products produces forty thousand pages with the same paragraph structure, the same transitions and the same closing sentence, differing only in the attributes interpolated. Those pages compete with each other for the same queries, and a store that previously had a hundred pages worth indexing now has forty thousand pages of which none is distinctive. That is a self-inflicted cannibalisation problem before it is anything to do with a search engine’s view of automation — the marketing page works through the structural version of this.

The second is claims. A generated description that says waterproof about a water-resistant watch, hypoallergenic about a nickel-plated earring, or dishwasher-safe about a hand-wash pan is a product claim the store is answerable for. The failure is not rare — the model is completing a pattern from the category, not reading the spec — and it is expensive in exactly the way returns and consumer complaints are. The control is mechanical: a claim vocabulary the generator is forbidden to use unless the corresponding attribute is present and positive in the record, enforced after generation as a check rather than requested in the prompt. Validation and repair covers the loop.

The experiment, and the metric that ruins it

Whether any of this earned money is answerable, and it needs an experiment because seasonality and merchandising changes swamp everything at the level of a before-and-after comparison. Two things decide whether that experiment is worth running.

• Randomise the shopper, not the query. The same person must get the same experience across a session and across devices where you can identify them, or the comparison measures confusion.
• The metric is revenue per session, not click-through. This is the trap. A ranker that promotes cheap, heavily discounted, attractively photographed items raises click-through and add-to-cart reliably and can lower revenue at the same time. If the team optimises on engagement it will ship that change and celebrate it. Pair revenue per session with a returns-adjusted version measured after the return window closes, because the other way to raise immediate revenue is to sell people things that come back.

For search specifically, an offline judgement set on the head queries — a merchandiser rating the top ten results for the two hundred queries that make most of the volume — is worth building first, because it turns a two-week online test into a ten-minute check for most changes and catches the regressions that would have wasted the test. Online for the decision, offline for the iteration, as in online evaluation.