Recommendation Systems: Collaborative vs Content-Based

The two approaches are usually compared on accuracy, which is the least useful axis, because their accuracy depends entirely on how much interaction data you have. Compare them on what they need as input and the choice mostly makes itself.

Two families, two inputs

Collaborative filtering uses the interaction matrix and nothing else. It knows which users touched which items and infers everything from the pattern. It does not know what the items are, and this is its strength: it will discover that two products go together for reasons no attribute in your catalogue captures.

Content-based filtering uses item attributes and nothing else. It builds a representation of each item from its metadata, text or images, builds a user profile from the items that user liked, and scores by similarity. It needs no other users at all, which means it works on day one and works for an item nobody has touched yet.

Item-item collaborative filtering, worked

Item-item is the workhorse rather than user-user, for a reason that is operational rather than statistical: item-item similarities are stable over hours or days and can be precomputed in batch, while a user’s neighbourhood changes with every click. Linden, Smith and York (2003) described the version Amazon ran on, and the shape has barely changed.

Take a binary interaction matrix — four users, three items, a tick meaning the user interacted:

         item1   item2   item3
user A     x       x
user B     x       x
user C     x               x
user D                     x

n(item1) = 3    n(item2) = 2    n(item3) = 2

Cosine similarity between two items, on binary vectors, is the number of users they share divided by the square root of the product of their counts:

sim(i, j) = co(i, j) / sqrt(n_i * n_j)

sim(1,2) = 2 / sqrt(3 * 2) = 2 / 2.449 = 0.816
sim(1,3) = 1 / sqrt(3 * 2) = 1 / 2.449 = 0.408
sim(2,3) = 0 / sqrt(2 * 2) = 0.000

Item 1 and item 2 are strongly related, item 1 and item 3 weakly, item 2 and item 3 not at all. A user who interacts with item 2 gets item 1 recommended, and nothing else. That is the entire algorithm, and it runs on a database with no machine learning in it whatsoever.

Matrix factorisation

The other classical approach learns a low-dimensional vector per user and per item so that their dot product reconstructs the observed ratings. The formulation popularised by Koren, Bell and Volinsky (2009) in their account of the Netflix Prize adds the bias terms that turn out to carry a surprising amount of the predictive power:

r_hat(u, i) = mu + b_u + b_i + dot(q_i, p_u)

  mu     global mean rating
  b_u    this user's tendency to rate high or low
  b_i    this item's tendency to be rated high or low
  p_u    user factor vector, learned
  q_i    item factor vector, learned

trained by minimising  SUM (r - r_hat)^2 + lambda * (norms)

The bias terms are worth keeping in mind because they are a free baseline: mu + b_u + b_i with no factors at all already predicts a large share of the variance in most rating data, and any model you build should be compared against it before you believe it works. The same idea reappears as a shrinkage prior in the cold-start page.

Modern systems have mostly replaced the explicit factorisation with a learned two-tower encoder, which generalises it — the factors become the output of a network that can also read features. That is the two-tower architecture, and it is the same objective with a richer function class.

The popularity trap

Here is the failure that every collaborative system has and that no amount of model capacity fixes. Raw co-occurrence co(i, j) is largest for the most popular items, simply because they appear in more baskets. Recommend by raw co-occurrence and every item recommends the bestseller — the classic illustration being that in the early 2000s a great many book recommenders recommended Harry Potter, from every starting point, because almost everyone had bought it.

The cosine denominator is the correction: dividing by sqrt(n_i * n_j) penalises popularity by the square root of the count. Sometimes that is not enough, and the generalised form gives you a dial:

sim(i, j) = co(i, j) / (n_i^alpha * n_j^(1 - alpha))

  alpha = 0.5  is exactly cosine
  alpha > 0.5  penalises the popularity of the SOURCE item more
  alpha -> 1   approaches a conditional probability P(j | i)

Whether the correction is working is not a modelling question, it is a measurement one: compute the share of your recommendation impressions that go to the top 1% of items by popularity, and watch it over time. If that number rises after a model change, the model got worse at the job whatever the offline metric said, because you have moved closer to a bestseller list — and a bestseller list does not need a recommendation system. Aggregate diversity and catalogue coverage belong on the same dashboard as accuracy; the mechanism for buying them back is in result diversification.

Choosing, and the answer being both

The honest answer for almost every real system is a hybrid, staged by how much you know:

• No interactions at all — content-based, or a popularity prior segmented by whatever context you have. Nothing else is available.
• Sparse interactions — content-based as the backbone, collaborative blended in with a weight proportional to the confidence of the neighbourhood. Confidence here means the number of co-occurring users, and it is worth computing explicitly rather than assuming.
• Dense interactions — collaborative as the backbone, content-based features folded in as inputs to the model rather than as a separate scorer. At this point the distinction has mostly dissolved, which is what a two-tower model with content features actually is.
• Always — a candidate-generation stage that is deliberately broader than the ranking stage, so the ranker has something other than the popular items to choose between. This is the same cascade argument as retrieve, rank, rerank, and recommendation converged on it independently.