Clustering

A cluster is a subset of data whose members are similar to one another. Clustering — the canonical form of unsupervised learning — divides a dataset into groups so that members of each group are as similar (close) as possible to one another, while different groups are as dissimilar (far) as possible. There are no labels and no target column: the structure itself is the discovery.

Because it needs no supervision, clustering is often the first algorithm run on a new dataset. It uncovers relationships nobody thought to label in advance: customer segments for marketing, patient subtypes in medicine, gene-expression groups in biology, communities in networks.

Measuring similarity

The central design decision in clustering is how to measure similarity between two objects, so clusters form from objects close together and far from other clusters. In practice, similarity is measured indirectly through a distance function, which returns smaller values for more similar pairs. For numerical vectors, the workhorses are the Minkowski family:

$D_{Euclidean} = \sqrt{\sum_{i=1}^{p}(x_i - y_i)^2}, \qquad D_{Manhattan} = \sum_{i=1}^{p} |x_i - y_i|, \qquad D_{Minkowski} = \left( \sum_{i=1}^{p} |x_i - y_i|^q \right)^{1/q}$

Euclidean distance ($q = 2$) is the default; Manhattan distance ($q = 1$) is more robust to outliers and behaves better in high dimensions. Whatever the choice, scale the features first — distance is geometry, and geometry depends on units.

Two families of algorithms

Hierarchical clustering builds a tree of nested clusters. Agglomerative methods start with each point in its own cluster and merge the closest pair repeatedly; divisive methods start with everything in one cluster and split recursively. The result is a dendrogram you can cut at any height — see hierarchical clustering.

Partitional clustering divides the data directly into $k$ flat groups. k-means iterates an assign-and-update loop to minimize within-cluster variance; self-organizing maps fold high-dimensional data onto a topology-preserving grid, doubling as a visualization tool.

What makes a good clustering method

• The ability to discover some or all of the hidden clusters.
• Within-cluster similarity and between-cluster dissimilarity.
• The ability to deal with various attribute types (numerical, categorical, mixed).
• Robustness to noise and outliers.
• Scalability to high dimensionality and large datasets.
• Results that are interpretable and usable downstream.

No single algorithm wins on all six — k-means is fast and scalable but assumes spherical, equal-size clusters; hierarchical clustering reveals structure at every granularity but costs $O(n^2)$ memory; density-based methods (DBSCAN, not covered in this series) find arbitrary shapes but struggle with varying densities.

Applications

Cluster analysis shows up wherever unlabeled data accumulates: marketing (discover and characterize customer segments), biology (group plants, animals, or genes by their features), information retrieval (cluster documents or search results), image analysis (segment pixels by color and position), and anomaly detection (points that belong to no cluster are worth a look). Clustering is also a feature-engineering step: cluster IDs and distances to centroids become inputs to a downstream supervised model.

In practice

sklearn.cluster covers the spectrum: KMeans (with kmeans++ initialization), AgglomerativeClustering, DBSCAN, and more. Because clustering has no ground truth, evaluate with internal metrics — silhouette score, Calinski–Harabasz, Davies–Bouldin — and, above all, with downstream utility: does the segmentation change a decision? Always try several algorithms and several values of $k$; clustering is exploratory, and the “right” answer is the one that survives scrutiny.

Common pitfalls

• Unscaled features, which let the largest-unit variable define “similarity” on its own.
• Trusting the first clustering — different algorithms and seeds give different answers; stability across runs is the weakest form of validation you must have.
• Reading clusters as facts — a clustering algorithm will find groups even in uniform noise. Always check whether the structure is real (silhouette, gap statistic, holdout stability).
• Choosing $k$ arbitrarily; sweep it and compare quality curves.
• Ignoring outliers, which drag centroids and distort merges; consider removing or modeling them separately.

Summary

Clustering partitions unlabeled data so that similar points land together and dissimilar points land apart, with a distance function defining “similar.” Hierarchical methods grow a nested tree of clusters; partitional methods like k-means and SOMs divide the space directly. The rest of this group works through each family: hierarchical clustering, k-means, and self-organizing maps.