Olivetti Faces

400 face images, 64×64 grayscale, k=40 identities — 10 examples each.

images

small

pixels

high-k

This is the notebook where DYF loses — and loses badly. That’s why it’s in the gallery. A parameter-free tool that’s only shown winning is a sales pitch, not an honest benchmark.

Olivetti faces is the anti-MNIST: small n (400 samples), huge d (4,096 pixels), high k (40 people, 10 photos each). Everything about it is hostile to density-based clustering. If DYF is going to break, it should break here — and understanding why is more valuable than pretending it doesn’t.

Load the data

Code

from sklearn.datasets import fetch_olivetti_faces

ol = fetch_olivetti_faces()
X = ol.data.astype("float32")   # (400, 4096) — 64×64 pixel vectors
y = ol.target                   # (400,) — 0..39, each person gets 10 photos

print(f"{X.shape[0]} samples, {X.shape[1]}d, true k = {len(set(y))}")
print(f"Examples per class: {len(X) // len(set(y))}")

400 samples, 4096d, true k = 40
Examples per class: 10

Run DYF

Code

import sys
sys.path.insert(0, ".")
from _gallery import run_dyf, run_kmeans, plot_single, metrics_table

result = run_dyf(X, y)
print(f"Recovered k: {result.recovered_k}  (true k = {result.true_k})")
print(f"NMI:         {result.nmi:.3f}")
print(f"ARI:         {result.ari:.3f}")

Recovered k: 3  (true k = 40)
NMI:         0.199
ARI:         0.037

That’s a blowout. DYF recovered a small handful of clusters where the ground truth has 40.

Oracle k-means

Code

kmeans = run_kmeans(X, y)
print(f"k-means NMI={kmeans['nmi']:.3f}, ARI={kmeans['ari']:.3f} (told k={result.true_k})")

k-means NMI=0.629, ARI=0.430 (told k=40)

Figures

Code

plot_single(result.umap_2d, y, title=f"Ground truth (k={result.true_k})")

Figure 1: Ground truth — 40 people, 10 photos each.

Code

plot_single(result.umap_2d, result.labels, title=f"DYF (recovered k={result.recovered_k})")

Figure 2: DYF — under-partitions severely.

Code

plot_single(result.umap_2d, kmeans["labels"], title=f"k-means (told k={result.true_k})")

Figure 3: K-means given the oracle k=40.

Metrics

Code

print(metrics_table(result, kmeans, None))

Method	Recovered k	NMI	ARI	% discarded
DYF (parameter-free)	3	0.199	0.037	0%
k-means (oracle k=40)	40	0.629	0.430	0%

Why DYF fails here

Three things are working against it simultaneously:

1. Not enough density per latent cluster. DYF’s tree splits recursively while each leaf contains more than min_leaf_size=20 points. With 40 identities and only 10 photos each, there is no way to carve out a leaf that isolates a single identity — the minimum leaf is 2× larger than any one class’s support. Louvain then runs on leaf-centroid similarity and merges related leaves into a handful of coarse communities. The floor is built into the algorithm.

You can fix this by lowering min_leaf_size, but that’s tuning a parameter — exactly what this gallery is showing DYF can avoid when it has enough data. The gallery’s DYF defaults are deliberately never tweaked.

2. n << d geometry. 400 points living in 4,096 dimensions is an extraordinary amount of empty space. Distances lose contrast. LSH-based splits are essentially finding random hyperplanes through a void; the “learned” PCA directions explain little of the real variance because 400 samples can only support ~400 meaningful principal components, and most of the 4,096 dimensions are pure noise for this sample size.

3. Face-identity is a curved manifold, not a density cluster. Even with unlimited photos per person, identity manifolds in raw pixel space are narrow and curved — lighting, pose, expression all move you along the manifold without changing identity. Density-based methods look for dense blobs surrounded by empty space; identity manifolds don’t look like that.

What the oracle gets you

K-means with k=40 handed over hits NMI 0.629, ARI 0.430. It’s a respectable number despite all the challenges — because with the oracle, k-means just has to find 40 centers. It doesn’t have to discover that there are 40 centers. That discovery is the hard part, and it’s what DYF is being asked to do unaided.

Would anything fix DYF here?

Yes — but the fixes are the kind of engineering work the gallery is trying to avoid as a user burden:

Pre-embed the faces with a face-recognition model. FaceNet / ArcFace embeddings compress each identity to a tight 128d or 512d cluster that density-based methods can find. This is the RAG-for-images pattern — embed with a model trained for the geometry you want, then cluster.
Lower min_leaf_size and let Louvain over-split. Works, but it’s tuning.
Use a different method. K-means with the oracle k wins here, and that’s fine. No tool dominates every regime.

The sports analogy

If Digits was k-means’s home court and 20 Newsgroups was an away game, Olivetti is a forfeit. DYF didn’t show up with the right equipment. It’s honest to say so.

What to take away

DYF has a floor condition: density per latent cluster. When your data has fewer than ~min_leaf_size × 2 samples per class, DYF will under-partition. That’s the pragmatic rule.
High-k / low-n is the hardest regime for any unsupervised method. Even k-means with the oracle only hits NMI 0.63 here; without the oracle it would do worse.
Dimensionality by itself isn’t the problem — 20 Newsgroups was 384d with 18K points and DYF tied k-means. n / d ratio and density per cluster are what matter.
If you can pre-embed with a model trained on your identity axis, the problem transforms into a geometry DYF handles well. That’s the right engineering move, and it’s what the DYF ecosystem is designed to compose with.

Caveats

This notebook deliberately uses raw pixels. A version with FaceNet embeddings would tell a very different story and probably match or beat k-means. The gallery shows the default behavior; it’s not claiming DYF cannot be helped with the right upstream model.
Olivetti is a 1994 benchmark dataset. Modern face-identity clustering is a solved problem with pretrained embeddings; this notebook is pedagogical, not a recommendation.

--- title: "Olivetti Faces" subtitle: "400 face images, 64×64 grayscale, k=40 identities — 10 examples each." order: 5 image: olivetti-faces_files/figure-html/fig-dyf-output-1.png categories: [images, small, pixels, high-k] true-k: 40 format: html: code-fold: show code-tools: true execute: warning: false message: false --- This is the notebook where DYF loses — and loses badly. That's why it's in the gallery. A parameter-free tool that's only shown winning is a sales pitch, not an honest benchmark. Olivetti faces is the *anti-MNIST*: small `n` (400 samples), huge `d` (4,096 pixels), high `k` (40 people, 10 photos each). Everything about it is hostile to density-based clustering. If DYF is going to break, it should break here — and understanding *why* is more valuable than pretending it doesn't. ## Load the data ```{python} #| label: load from sklearn.datasets import fetch_olivetti_faces ol = fetch_olivetti_faces() X = ol.data.astype("float32") # (400, 4096) — 64×64 pixel vectors y = ol.target # (400,) — 0..39, each person gets 10 photos print(f"{X.shape[0]} samples, {X.shape[1]}d, true k = {len(set(y))}") print(f"Examples per class: {len(X) // len(set(y))}") ``` ## Run DYF ```{python} #| label: run-dyf import sys sys.path.insert(0, ".") from _gallery import run_dyf, run_kmeans, plot_single, metrics_table result = run_dyf(X, y) print(f"Recovered k: {result.recovered_k} (true k = {result.true_k})") print(f"NMI: {result.nmi:.3f}") print(f"ARI: {result.ari:.3f}") ``` That's a blowout. DYF recovered a small handful of clusters where the ground truth has 40. ## Oracle k-means ```{python} #| label: kmeans kmeans = run_kmeans(X, y) print(f"k-means NMI={kmeans['nmi']:.3f}, ARI={kmeans['ari']:.3f} (told k={result.true_k})") ``` ## Figures ::: {.gallery-fluid} ```{python} #| label: fig-truth #| fig-cap: "Ground truth — 40 people, 10 photos each." plot_single(result.umap_2d, y, title=f"Ground truth (k={result.true_k})") ``` ```{python} #| label: fig-dyf #| fig-cap: "DYF — under-partitions severely." plot_single(result.umap_2d, result.labels, title=f"DYF (recovered k={result.recovered_k})") ``` ```{python} #| label: fig-kmeans #| fig-cap: "K-means given the oracle k=40." plot_single(result.umap_2d, kmeans["labels"], title=f"k-means (told k={result.true_k})") ``` ::: ## Metrics ```{python} #| label: metrics #| output: asis print(metrics_table(result, kmeans, None)) ``` ## Why DYF fails here Three things are working against it simultaneously: **1. Not enough density per latent cluster.** DYF's tree splits recursively while each leaf contains more than `min_leaf_size=20` points. With 40 identities and only 10 photos each, there is no way to carve out a leaf that isolates a single identity — the minimum leaf is 2× larger than any one class's support. Louvain then runs on leaf-centroid similarity and merges related leaves into a handful of coarse communities. The floor is built into the algorithm. You can fix this by lowering `min_leaf_size`, but that's *tuning a parameter* — exactly what this gallery is showing DYF can avoid when it has enough data. The gallery's DYF defaults are deliberately never tweaked. **2. `n << d` geometry.** 400 points living in 4,096 dimensions is an extraordinary amount of empty space. Distances lose contrast. LSH-based splits are essentially finding random hyperplanes through a void; the "learned" PCA directions explain little of the real variance because 400 samples can only support ~400 meaningful principal components, and most of the 4,096 dimensions are pure noise for this sample size. **3. Face-identity is a curved manifold, not a density cluster.** Even with unlimited photos per person, identity manifolds in raw pixel space are narrow and curved — lighting, pose, expression all move you along the manifold without changing identity. Density-based methods look for dense blobs surrounded by empty space; identity manifolds don't look like that. ### What the oracle gets you K-means with `k=40` handed over hits NMI 0.629, ARI 0.430. It's a respectable number despite all the challenges — because with the oracle, k-means just has to find 40 centers. It doesn't have to discover that there *are* 40 centers. That discovery is the hard part, and it's what DYF is being asked to do unaided. ### Would anything fix DYF here? Yes — but the fixes are the kind of engineering work the gallery is trying to avoid as a user burden: - **Pre-embed the faces with a face-recognition model.** FaceNet / ArcFace embeddings compress each identity to a tight 128d or 512d cluster that density-based methods *can* find. This is the RAG-for-images pattern — embed with a model trained for the geometry you want, then cluster. - **Lower `min_leaf_size` and let Louvain over-split.** Works, but it's tuning. - **Use a different method.** K-means with the oracle *k* wins here, and that's fine. No tool dominates every regime. ### The sports analogy If Digits was k-means's home court and 20 Newsgroups was an away game, Olivetti is a *forfeit*. DYF didn't show up with the right equipment. It's honest to say so. ## What to take away - **DYF has a floor condition: density per latent cluster.** When your data has fewer than ~`min_leaf_size × 2` samples per class, DYF will under-partition. That's the pragmatic rule. - **High-k / low-n is the hardest regime** for any unsupervised method. Even k-means with the oracle only hits NMI 0.63 here; without the oracle it would do worse. - **Dimensionality by itself isn't the problem** — 20 Newsgroups was 384d with 18K points and DYF tied k-means. `n / d` ratio and density per cluster are what matter. - **If you can pre-embed with a model trained on your identity axis**, the problem transforms into a geometry DYF handles well. That's the right engineering move, and it's what the DYF ecosystem is designed to compose with. ## Caveats - This notebook deliberately uses raw pixels. A version with FaceNet embeddings would tell a very different story and probably match or beat k-means. The gallery shows the *default* behavior; it's not claiming DYF cannot be helped with the right upstream model. - Olivetti is a 1994 benchmark dataset. Modern face-identity clustering is a solved problem with pretrained embeddings; this notebook is pedagogical, not a recommendation.