Clustering That Produces Segments People Use

Ask k-means for five clusters on pure noise and it returns five clusters, each with a centroid, each with a silhouette score, each ready to be given a name in a slide. Clustering has no held-out set and no ground truth, which makes validation the whole discipline — and it is the step almost every segmentation project omits.

The algorithm always returns clusters

Unsupervised methods cannot fail visibly. There is no accuracy that drops, no loss that stalls. Every partitioning algorithm applied to every dataset produces a partition, and the partition always admits a narrative: the high-value engaged group, the price-sensitive occasionals, the dormant.

So the question a clustering project must answer is not “what are the segments” but “is there any structure here at all, and would the same structure appear in a different sample of the same population”. Two tests answer it, and both are below.

Four methods and what each assumes

The noise label is the underrated feature. A segmentation that assigns every customer to a segment is making claims about customers who do not resemble any group, and those claims are the ones that fail when the marketing team acts on them.

Preparation decides the result

1. Scale, or the largest unit wins. Every method here except the tree-based ones uses Euclidean distance, so a column in pounds dominates a column in counts entirely. Standardise, and prefer a robust scaler when the features are skewed — which spend, sessions and order counts always are.
2. Transform the skew before scaling. A log or square-root transform on monetary and count features does more for cluster quality than any algorithm choice. Without it, the top 1% of spenders become their own cluster and everything else becomes one blob.
3. Choose few features, deliberately. Distance becomes uninformative in high dimensions — every pair of points ends up roughly equidistant. Five to fifteen well-chosen features beat fifty, and the choice is a domain decision about what the segmentation is for.
4. Handle correlated features. Six variants of “how much they spend” is six votes for the same dimension, silently weighting it six times. Either drop the duplicates or reduce with PCA first and accept the loss of interpretability — the same judgement feature engineering calls for on the supervised side.
5. Decide about outliers explicitly. k-means centroids are means and are dragged by them. Either remove them, use a density-based method that labels them noise, or accept that one cluster will be four unusual customers.

Silhouette, worked, and its bias

The silhouette of a point compares how close it sits to its own cluster against how close it sits to the nearest other cluster.

For a point i:

  a(i) = mean distance from i to the other points in ITS cluster
  b(i) = mean distance from i to the points of the NEAREST other cluster

  s(i) = (b - a) / max(a, b)

Worked, three points:

  point   a      b      s
    p    2.0    5.0    (5.0-2.0)/5.0 = +0.60    comfortably inside
    q    4.0    4.4    (4.4-4.0)/4.4 = +0.09    on the boundary
    r    5.0    3.0    (3.0-5.0)/5.0 = -0.40    closer to another cluster

  s ranges from -1 to +1. The silhouette score of a clustering is the
  mean of s(i) over all points.

Conventional reading:
  above 0.5   reasonable separation
  0.25-0.5    weak, overlapping structure
  below 0.25  no meaningful separation, whatever the picture looks like

Now the caveat that is almost never stated. Silhouette is built on compact, well-separated, roughly convex clusters — which is exactly what k-means produces. So it systematically favours k-means output and systematically penalises the elongated or irregular clusters that density-based methods find, even when those are the true structure. Comparing HDBSCAN against k-means on silhouette is a rigged comparison.

Use it to choose k within a method, never to choose between methods. For that, use the stability test.

The stability test

Real structure reproduces on a different sample of the same population. Structure the algorithm invented does not. That is directly testable and takes about twenty lines.

1. Cluster the full dataset. Keep the labels.
2. Draw a bootstrap resample — n rows with replacement — and cluster it with identical settings.
3. On the rows that appear in both, compare the two labellings with the adjusted Rand index, which measures agreement corrected for the agreement expected by chance.
4. Repeat twenty times and take the mean and spread.

Conventional reading of mean ARI across resamples:

  above 0.75    stable; the structure is reproducible
  0.60 - 0.75   tentative; report it as provisional
  below 0.60    not reproducible; the clusters are an artefact of
                this sample and should not be given names

These bands are a working convention in the cluster-validation
literature, not a theorem. Report the number itself alongside any
label, so the reader can apply their own bar.

Run this at several values of k. A common and useful result is that k = 3 is highly stable while k = 7 is not, which means there are three real groups and four narratives. Choosing k by silhouette alone would often have picked the seven.

Does the segmentation predict anything

Stability says the clusters are real. It does not say they are worth anything. The business test is whether cluster membership predicts an outcome the clusters were not built from.

• Hold out an outcome. Cluster on behavioural features only, then check whether segment membership predicts churn, expansion revenue or support cost on a later period — scored the way the churn page scores a rare event. If it does not, you have a description rather than a segmentation.
• Check that the segments differ on something actionable. Two segments that differ only on features nobody can influence are one segment for practical purposes.
• Check the sizes. A segmentation with a 94% cluster and four small ones has told you that your customers are mostly alike, which is a genuine finding and not the one anyone wanted.
• Check persistence. Re-run next quarter and count how many customers changed segment. If half of them move, the segments describe a moment rather than a type, and any campaign built on them is targeting last quarter’s people.

The whole procedure

import numpy as np
from sklearn.cluster import KMeans
from sklearn.metrics import adjusted_rand_score, silhouette_score
from sklearn.preprocessing import RobustScaler

Xs = RobustScaler().fit_transform(np.log1p(X))     # skew, then scale

def stability(Xs, k, n_boot=20, seed=0):
    rng = np.random.default_rng(seed)
    base = KMeans(n_clusters=k, n_init=10, random_state=0).fit_predict(Xs)
    scores = []
    for _ in range(n_boot):
        idx = rng.choice(len(Xs), size=len(Xs), replace=True)
        uniq = np.unique(idx)                       # rows present in both
        lab = KMeans(n_clusters=k, n_init=10,
                     random_state=0).fit_predict(Xs[idx])
        # label for each unique row: take its first occurrence in the sample
        first = {r: lab[np.where(idx == r)[0][0]] for r in uniq}
        scores.append(adjusted_rand_score(base[uniq],
                                          np.array([first[r] for r in uniq])))
    return float(np.mean(scores)), float(np.std(scores))

for k in range(2, 11):
    labels = KMeans(n_clusters=k, n_init=10, random_state=0).fit_predict(Xs)
    sil = silhouette_score(Xs, labels)
    m, s = stability(Xs, k)
    print(f"k={k:2d}  silhouette={sil:.3f}  ARI={m:.3f} +/- {s:.3f}")

Read the two columns together. High silhouette with low ARI means the algorithm found a tidy partition of noise. Moderate silhouette with high ARI means overlapping but genuine groups, which is what real customer data usually looks like and is a perfectly good result.

For HDBSCAN, use sklearn.cluster.HDBSCAN (available in scikit-learn 1.3 and later) or the standalone hdbscan package, and score stability the same way while excluding the noise label from the comparison. Check the parameter names against your installed version; the two implementations differ in places.

One more caution specific to small datasets. Below a couple of thousand rows the bootstrap ARI is itself noisy, and the sample-size arithmetic applies here as much as it does to a classifier. For text, the same machinery over vectors is clustering over embeddings, where the preparation steps above are replaced by the choice of model and the number of dimensions kept.