Case file
Can 569 breast fine-needle aspirates, each described by 30 nuclear measurements, be separated into benign and malignant without ever being told which is which? Without labels the data splits in two, and the split matches the diagnosis 92% of the time. It still misses one cancer in seven, and the case file shows which ones.
Thesis
Can 569 breast fine-needle aspirates, each described by 30 nuclear measurements, be separated into benign and malignant without ever being told which is which? If the data sorts itself, the structure is real and not an artefact of the labels. And when the sorting fails, the cases it misses say something about where the two groups overlap.
What the 2023 exercise lacked
The course exercise ran PCA in MATLAB and k-means and hierarchical clustering in Python on the Iris flower dataset. It had no question, no reason for the number of clusters, no measure of agreement, no test of stability and no check on whether a different reasonable pipeline would have given a different answer. This applies the same methods to medical data and adds each of those parts.
Method
Log transform, then a z-score, on all 30 features, because area and perimeter are strongly right-skewed. PCA as in the course: 10 components explain 95% of the variance and PC1 alone explains 45%. The number of clusters is chosen without labels, by silhouette over k = 2 to 8. k = 2 wins. The primary analysis is k-means with k = 2 on the full standardised space with 20 initialisations. Only then is the diagnosis revealed. Each cluster is named after its majority class and scored. A sensitivity analysis runs every combination of raw or log features, all features or 2 or 95% principal components, and k-means, Ward or a Gaussian mixture: 18 pipelines, all reported.
Result
Without labels the data splits in two, and the two groups line up with the diagnosis 92% of the time. Adjusted Rand index 0.71. The split is identical across 100 random seeds and holds at a median ARI of 0.97 across 200 bootstrap resamples.
It still misses one cancer in seven. 32 of 212 malignant tumours land in the benign cluster. A supervised logistic regression on the same features catches 96% of cancers under 10-fold cross-validation. A label-free grouping is a description of the data, not a screening tool.
Which cancers it misses
The 32 missed malignancies sit between the two groups. Their nuclei are smaller and less concave than the malignancies that are caught, with a median radius of 14.8 against 18.0 and median worst area of 892 against 1431. Their texture is already malignant: a median of 21.5 in both groups, against 17.4 for benign cases.
Clustering splits the data along the dominant axis of variation. That axis is PC1, driven by concavity, concave points, compactness and perimeter. Mean texture ranks 25th of 30 on it. So the cancers clustering misses are the small, smooth-edged ones, where size and shape have not yet diverged from benign. Unsupervised structure follows the biggest variance, not the clinically important one.
Engineering decisions
Fix the primary analysis before looking at the labels. Across the 18 pipelines, agreement with diagnosis ranges from ARI 0.03 to 0.77, and one pipeline scores higher than the primary one. Picking the best after seeing the labels would let the labels choose the method. The primary choice is fixed on label-free grounds: the course method, a log transform for the skew, and k from the silhouette.
Publish every pipeline. I had looked at an exploratory version of the sensitivity table before fixing the primary choice. That is why all 18 results are shown rather than the one that was chosen.
Test that the labels never reach the clustering. One test asserts the cluster assignment is unchanged when the diagnosis column is shuffled or removed. Others check the dataset shape, the cluster-to-diagnosis mapping and error counts, and that the headline numbers reproduce.
Build status
Done. The analysis runs in about 30 seconds, writes a results file and three figures, and the tests pass from a clean checkout.
Limits. One public dataset from one institution, Wisconsin in the early 1990s, with features extracted by the original authors' software. The diagnosis is treated as ground truth. This is an analysis exercise, not a diagnostic or screening tool.
Credit
2023 team: Jeremia Kerstner, Manuel Kalus, Robert Aigner and Samuel Henry Mba, FH Technikum Wien, course Advanced Analysis of Medical Data. The 2026 analysis on WDBC is mine. Data: Wolberg, Mangasarian, Street and Street, Breast Cancer Wisconsin (Diagnostic), UCI Machine Learning Repository, 1993, doi:10.24432/C5DW2B, CC BY 4.0.