CIFAR-10 + CLIP

10,000 natural images, 512d CLIP embeddings — explore DYF’s partition hierarchy interactively.

import os, sys, time
import numpy as np

sys.path.insert(0, ".")
from datasets import load_dataset
from _gallery import (
    run_dyf_cached, run_kmeans, hierarchy_slider,
    merge_walk, merge_walk_table, metrics_table, plot_single,
)

# 1. Load CIFAR-10 (test split, 10K images, 10 classes).
ds = load_dataset("cifar10", split="test")
CLASS_NAMES = ds.features["label"].names

# 2. Embed via CLIP ViT-B/32 (or load cache).
CACHE = "/tmp/gallery_cifar10_clip.npz"
if os.path.exists(CACHE):
    z = np.load(CACHE)
    X, y = z["X"], z["y"]
else:
    from sentence_transformers import SentenceTransformer
    model = SentenceTransformer("clip-ViT-B-32")
    imgs = [d["img"] for d in ds]
    y = np.asarray([d["label"] for d in ds])
    X = np.ascontiguousarray(
        model.encode(imgs, show_progress_bar=False, batch_size=64),
        dtype=np.float32,
    )
    np.savez(CACHE, X=X, y=y)

# 3. Fit DYF + k-means (cached).
result = run_dyf_cached(X, y, cache_path="/tmp/gallery_cifar10_dyf.npz")
kmeans = run_kmeans(X, y)

Explore the hierarchy

Drag the slider. Each position is a different cut of DYF’s partition tree — same data, same fit, different resolution. The left panel shows DYF’s partition at the current k; the right panel shows the ground-truth class labels on the same UMAP layout. Hover a point to see its true class. Watch the AMI / ARI numbers in the title move as you change resolution.

fig = hierarchy_slider(
    result, X, y,
    k_values=[11, 10, 8, 6, 4, 3, 2],
    class_names=CLASS_NAMES,
    show_ground_truth=True,
    height=500,
)
fig

DYF’s partition (left, slider-controlled) vs ground-truth classes (right, fixed). Hover for class names.

Three things to try:

• Start at k=2 (default). The partition collapses to two clusters — in our runs, roughly vehicles (airplane/car/ship/truck) vs animals; hover to check where the boundary falls. A superclass split emerging from raw CLIP similarity alone.
• Slide to k=11. This is DYF’s own finest partition — eleven communities for ten classes, recovered without being told the number. One step left, at k=10, is the label’s own count; both positions beat the oracle-tuned k-means below.
• Watch what merging costs. Every step left of 11 drops both metrics; the flat label sits one merge below the resolution DYF stopped at.

K-means would need a separate run at every k to produce this range — and those runs are not guaranteed to nest. DYF’s tree nests by construction: every coarser partition is a strict merge of a finer one.

How we got here

The setup code is collapsed above. The short version:

1. Load CIFAR-10’s test split (10,000 images, 10 classes).
2. Embed every image with off-the-shelf CLIP ViT-B/32 — no fine-tuning — producing 512-dim vectors that put semantically similar images into tight blobs.
3. Run DYF with its standard defaults on those embeddings.
4. Run oracle-k-means (k=10) for comparison.

Why CLIP and not raw pixels? Because Olivetti showed what happens when raw pixels go straight into DYF: at 400 samples × 4,096d, DYF collapsed to 4 clusters against a 40-class ground truth and posted AMI=0.258. The diagnosis was that raw pixel space has bad geometry for density clustering — identity manifolds are curved and narrow, not the tight blobs density methods need. Swapping in a pretrained visual encoder fixes the geometry and gives DYF adequate sample density per latent class.

This notebook is the counter-narrative to Olivetti: images aren’t hostile to DYF; raw pixels are.

Metrics

print(metrics_table(result, kmeans, None))

Method	Recovered k	AMI	ARI	% discarded	wall (s)
DYF (parameter-free)	11	0.741	0.683	0%	0.91
k-means (oracle k=10)	10	0.686	0.515	0%	0.10

Wall time: DYF 0.91 s (tree build + index write + Louvain) vs k-means 0.10 s (one k-means++ init at the oracle k) — DYF 9.5× slower here. The shared UMAP layout is not counted for either.

On wall time DYF is the slower of the two, as on every gallery page; the wall (s) column says by how much. The hierarchy below is what the extra time buys.

Walking down the hierarchy (the table view)

The slider is interactive; here’s the same data as a static table for easy comparison across resolutions:

walk = merge_walk(result, X, y, targets=[15, 10, 5, 2])
print(merge_walk_table(walk, result))

Resolution	Actual k	AMI	ARI
Raw DYF	11	0.741	0.683
merge → 15	11	0.741	0.683
merge → 10	10	0.720	0.639
merge → 5	5	0.652	0.441
merge → 2	2	0.389	0.162

The raw partition is the k=11 row — AMI 0.741, ARI 0.683 against oracle-k k-means at 0.686 / 0.515 — and merge → 10 (0.720 / 0.639) still beats k-means at its own k. DYF found the 10-class structure, plus one split the label has no room for, without being told the number; the merge walk only goes down from there.

Static figures (for the record)

Same UMAP layout, three fixed views — ground truth, raw DYF, and k-means:

plot_single(result.umap_2d, y, title=f"Ground truth (k={result.true_k})")

Figure 1: Ground truth — 10 CIFAR classes (airplane, car, bird, cat, deer, dog, frog, horse, ship, truck).

plot_single(result.umap_2d, result.labels, title=f"DYF (recovered k={result.recovered_k})")

Figure 2: DYF — parameter-free; recovers the class count on its own.

plot_single(result.umap_2d, kmeans["labels"], title=f"k-means (told k={result.true_k})")

Figure 3: K-means given the oracle k=10.

The counter-narrative: images aren’t the problem, pixels are

Stack this alongside Olivetti:

Dataset	Input	n × d	DYF k	DYF AMI	KMeans AMI	DYF AMI − KMeans
Olivetti	Raw pixels	400 × 4096	4	0.258	0.629	−0.371
CIFAR-10	CLIP embeddings	10,000 × 512	11	0.741	0.686	+0.055

The DYF-vs-oracle-k-means gap went from a rout to a DYF win on both metrics (AMI 0.741 vs 0.686, ARI 0.683 vs 0.515) once the features changed. Same algorithm, same defaults, transformative result. Two things moved together:

1. The geometry changed. Raw face pixels don’t form tight identity clusters — lighting, pose, and expression spread each person across a long, curved manifold in 4,096-dimensional pixel space. CLIP embeddings, by contrast, were trained to put visually-distinct classes into tight blobs in 512-dimensional representation space. Density-based clustering needs “tight blobs separated by empty space” to work; CLIP gives you exactly that.

2. Density per cluster improved dramatically. Olivetti had 10 samples per identity — below DYF’s leaf-size floor. CIFAR-10 test has 1,000 samples per class, comfortably above it. Both changes mattered; separating their contributions would need a third experiment (e.g., CLIP on a 400-image subset), which we’re skipping for the gallery.

The honest conclusion: raw-pixel image clustering is a solved problem and the solution is “use a pretrained embedder.” DYF plays this game well once the inputs are the right shape; it doesn’t try to solve pixel-space geometry from scratch, and you shouldn’t expect it to.

The recipe

Looking across the gallery, a practical recipe is emerging:

1. If your data has meaningful density structure in its raw feature space (motion capture joint angles, text sentence embeddings, tabular sensor streams) → DYF works out of the box.
2. If your raw features are bad geometry for density clustering (raw pixels, one-hot categoricals, unnormalized mixed-scale features) → pre-embed with a domain-appropriate model first, then run DYF.
3. If your data is nested / topological (concentric rings, donut shapes) → neither DYF nor k-means helps; reach for spectral clustering.

This is the same logic behind “put an embedding model in front of your retrieval / clustering / ranking pipeline” — the now-canonical move in RAG, recommender systems, and anomaly detection. DYF is a natural downstream consumer of pretrained embeddings, not a replacement for them.

Caveats

• Test split only (10,000 images). The full CIFAR-10 train split (50,000) would give DYF more density per class; we use the test split to keep the notebook render fast.
• CLIP was trained on internet-scale image-text pairs, which almost certainly included CIFAR-like images. It’s the “home court” of pretrained vision encoders. A domain-specific corpus (medical imaging, satellite, microscopy) would need a domain-specific encoder to tell the same story.
• No fine-tuning. The CLIP encoder is used off-the-shelf. Fine-tuning on CIFAR would tighten the clusters further. That’s also the recipe for “why does this method work so well on this benchmark” in a lot of published clustering papers.
• Earlier versions of this page reported 21–23 DYF clusters and a merge-walk that “found” the 10-class structure at merge → 10. The extra clusters were an artifact of a single-level Louvain in dyf-rs before 0.12.0 that could not merge communities (dyf KNOWN_ISSUES.md, issue 10). With the corrected optimiser the raw partition lands at 11, one merge above the label.