CMU MoCap

140,837 motion-capture frames, 62d joint angles — true k is ambiguous.

Every other gallery notebook has a clean ground-truth k — 10 digits, 20 newsgroup topics, 40 identities. CMU Motion Capture breaks that convenience. The same frames are simultaneously labeled with 7 motion types (walk, dance, climb, …), 25 descriptions (slow walk, fast walk, salsa, …), and 80 trial IDs (individual recording sessions). Which one is “true k”?

None of them. Or all of them. A dance trial contains walking phases. Climbing involves reaching. Running and walking share gait dynamics at transitions. The label hierarchy is a human convenience layered over continuous motion that refuses to partition cleanly at any single granularity.

That makes this the gallery’s most honest notebook: a dataset where no frame-level clustering — DYF’s or k-means’s — scores well against the labels, and the reason is instructive.

Load the data

Joint-angle features from the CMU Motion Capture Database — 62 channels per frame (31 joints × Euler ZYX angles), 120fps recordings across 80 trials. No embedding model. DYF operates directly on the physical signal.

import numpy as np
import polars as pl

FEATURES = "../data/cmu_mocap_features.npy"
META     = "../data/cmu_mocap_metadata.parquet"

X_full = np.load(FEATURES).astype(np.float32)
m_full = pl.read_parquet(META)

# Subsample to every 5th frame — 120fps mocap is heavily redundant frame-to-frame,
# and the gallery render needs to complete in a reasonable wall-clock budget.
stride = 5
idx = np.arange(0, len(X_full), stride)
X = X_full[idx]
m = m_full[idx.tolist()]
print(f"Full: {X_full.shape[0]:,} frames → subsampled ({stride}×): {X.shape[0]:,} frames × {X.shape[1]}d")
print(f"Motion types ({m['motion_type'].n_unique()}): {sorted(m['motion_type'].unique().to_list())}")

Full: 140,837 frames → subsampled (5×): 28,168 frames × 62d
Motion types (7): ['basketball', 'climb', 'dance', 'jump', 'run', 'sit_stand', 'walk']

Three ground truths

y_motion = m["motion_type"].to_numpy()
y_desc   = m["description"].to_numpy()
y_trial  = m["trial_id"].to_numpy()

_, y_motion_idx = np.unique(y_motion, return_inverse=True)
_, y_desc_idx   = np.unique(y_desc,   return_inverse=True)
_, y_trial_idx  = np.unique(y_trial,  return_inverse=True)

print(f"motion_type: {len(np.unique(y_motion_idx))} classes — broad categories")
print(f"description: {len(np.unique(y_desc_idx))} classes — finer movement types")
print(f"trial_id:    {len(np.unique(y_trial_idx))} classes — individual recording sessions")

motion_type: 7 classes — broad categories
description: 25 classes — finer movement types
trial_id:    80 classes — individual recording sessions

Run DYF (parameter-free) on the 62d joint angles

No embedding. The 62 Euler-angle channels are DYF’s direct input.

import sys
sys.path.insert(0, ".")
from _gallery import run_dyf_cached, run_kmeans, plot_single

# Score against the middle-granularity label (description, k=25) for the main run.
result = run_dyf_cached(X, y_desc_idx, cache_path="/tmp/gallery_mocap_dyf.npz")
print(f"Recovered k:       {result.recovered_k}")
print(f"AMI vs description: {result.ami:.3f}")
print(f"ARI vs description: {result.ari:.3f}")

Recovered k:       15
AMI vs description: 0.292
ARI vs description: 0.116

How well does DYF’s partition match each ground truth?

from sklearn.metrics import adjusted_mutual_info_score as ami
from sklearn.metrics import adjusted_rand_score as ari

rows = []
for name, y_idx in [("motion_type (k=7)",   y_motion_idx),
                    ("description (k=25)",  y_desc_idx),
                    ("trial_id (k=80)",     y_trial_idx)]:
    rows.append((name, ami(y_idx, result.labels), ari(y_idx, result.labels)))

print(f"{'Ground truth':25s}  AMI     ARI")
for name, m_ami, m_ari in rows:
    print(f"{name:25s}  {m_ami:.3f}   {m_ari:.3f}")

Ground truth               AMI     ARI
motion_type (k=7)          0.247   0.114
description (k=25)         0.292   0.116
trial_id (k=80)            0.332   0.088

Low across the board, and the pattern is telling: AMI rises with label granularity (7 → 25 → 80 classes) while ARI barely moves. The partition shares information with the trial structure but does not reproduce any level of it as clusters.

Oracle k-means on the middle granularity

kmeans = run_kmeans(X, y_desc_idx)
print(f"K-means told k=25: AMI={kmeans['ami']:.3f}, ARI={kmeans['ari']:.3f}  wall={kmeans['seconds']:.1f}s")
print(f"DYF (no k given):  AMI={result.ami:.3f}, ARI={result.ari:.3f}  wall={result.seconds:.1f}s")

K-means told k=25: AMI=0.348, ARI=0.103  wall=0.1s
DYF (no k given):  AMI=0.292, ARI=0.116  wall=0.3s

DYF edges k-means on ARI and trails it on AMI, and both are far from good. Being handed k=25 did not rescue k-means: the labels are not the kind of thing a frame-level partition can recover, whichever algorithm draws it.

Walking down the hierarchy against each ground truth

DYF’s recovered partition is multi-resolution by design — and CMU MoCap has three plausible “true k” values to aim at (motion_type=7, description=25, trial_id=80):

from _gallery import merge_walk, merge_walk_table

print("**Against `motion_type` (k=7):**\n")
print(merge_walk_table(merge_walk(result, X, y_motion_idx, targets=[25, 10, 7, 3]), result, y_true=y_motion_idx))
print("\n\n**Against `description` (k=25):**\n")
print(merge_walk_table(merge_walk(result, X, y_desc_idx, targets=[40, 25, 15, 7]), result, y_true=y_desc_idx))
print("\n\n**Against `trial_id` (k=80):**\n")
print(merge_walk_table(merge_walk(result, X, y_trial_idx, targets=[60, 40, 25, 10]), result, y_true=y_trial_idx))

Against motion_type (k=7):

Resolution	Actual k	AMI	ARI
Raw DYF	15	0.247	0.114
merge → 25	15	0.247	0.114
merge → 10	10	0.267	0.172
merge → 7	7	0.151	0.054
merge → 3	3	0.033	-0.039

Against description (k=25):

Resolution	Actual k	AMI	ARI
Raw DYF	15	0.292	0.116
merge → 40	15	0.292	0.116
merge → 25	15	0.292	0.116
merge → 15	15	0.292	0.116
merge → 7	7	0.179	0.033

Against trial_id (k=80):

Resolution	Actual k	AMI	ARI
Raw DYF	15	0.332	0.088
merge → 60	15	0.332	0.088
merge → 40	15	0.332	0.088
merge → 25	15	0.332	0.088
merge → 10	10	0.316	0.086

Read this vertically. DYF’s partition is the same across all three tables — only the ground-truth target changes — and merging does not help against any of them: collapsing to 7 does not recover the 7 motion types, and the raw partition scores highest against the 80 trials. If the frames grouped into nested blobs at the label granularities, the merge walk would find them. It does not, because they don’t.

Figures

plot_single(result.umap_2d, y_motion_idx, title="Ground truth: motion_type (k=7)")

Figure 1: Ground truth — motion_type (7 broad categories: walk, dance, climb, …).

plot_single(result.umap_2d, y_desc_idx, title="Ground truth: description (k=25)")

Figure 2: Ground truth — description (25 finer-grained movement types).

plot_single(result.umap_2d, result.labels, title=f"DYF (recovered k={result.recovered_k})")

Figure 3: DYF — parameter-free, found its own resolution.

plot_single(result.umap_2d, kmeans["labels"], title="k-means (told k=25)")

Figure 4: K-means given the oracle k=25 (description granularity).

Why frame-level clustering struggles here

Look at the ground-truth panels: a trial is not a blob, it is a path. A walking cycle sweeps through pose space and returns; a dance trial visits several regions in sequence. Two trials of the same description overlap where their paths cross and diverge where they don’t. Any method that partitions frames by where they sit in pose space — DYF’s leaves, k-means’s Voronoi cells — will group frames from different trials that happen to pass through the same pose, and split one trial across the regions its path visits. That is exactly the AMI-rises-with-granularity / ARI-stays-flat signature above.

Three consequences:

1. The labels are about sequence; the features are about position. The natural signal for “which trial is this” is adjacent-frame structure — consecutive frames are nearly identical, and a trial boundary is a jump. The raw frame-to-frame step distance separates trial boundaries almost perfectly on this data; a frame-level partition throws that information away by construction. A time-aware model (HMM, sequence embedding) is the right tool for the label; this notebook is deliberately about frame-level structure only.

2. Being told k does not help. K-means at the oracle k=25 lands in the same low range as DYF. When the label is not a partition of the feature space, the number of clusters is not the missing information.

3. What DYF’s 15 communities are. They are regions of pose space — standing poses, mid-stride poses, arms-raised poses — that several trials share. That is real structure, and for a retrieval task (“find frames that look like this one”) it is the right structure. It is simply not what the trial labels describe.

The sports analogy, completed

Across the four notebooks we now have:

Dataset	Geometry	DYF vs oracle k-means
Digits	Convex blobs, clean k, tiny tree	K-means wins (home court)
MNIST	Convex-ish, 40× more data	DYF wins on both metrics
20 Newsgroups	Curved, hierarchical	K-means ahead by 0.01 AMI
CMU MoCap	Trajectories, ambiguous k	Neither works; DYF edges ARI, trails AMI

The pattern isn’t “DYF vs k-means.” It is whether the label is a partition of the feature space at all. When it is, and the tree has enough leaves, DYF recovers it without being told k. When it isn’t — as here — no frame-level method will, and the honest move is to say so and reach for a sequence model.

What to take away

• “True k” is often a fiction. If a dataset has multiple plausible labelings at different granularities, your clustering algorithm shouldn’t demand you pick one upfront — but it also cannot recover a label that is not a partition of the features.
• DYF operates on raw features, no embedding required. Motion capture, tabular data, sensor readings — anywhere the raw features have meaningful geometry, DYF can go in direct.
• Read AMI and ARI together. AMI rising with label granularity while ARI stays flat is the fingerprint of shared information without shared clusters. On its own, either number would have told a different, wrong story.

Caveats

• Subsampled to 28,167 frames for the gallery render. The full 140,837-frame results show the same pattern.
• No temporal structure used. These are frame-level features — adjacent-frame similarity within a walk cycle isn’t exploited. That is the whole diagnosis above, not a footnote to it.
• Earlier versions of this page reported 56 communities and “DYF beats oracle k-means on AMI against every label.” The count came from a single-level Louvain in dyf-rs before 0.12.0 that could not merge communities (dyf KNOWN_ISSUES.md, issue 10), and the over-split partition inflated AMI against fine-grained labels. The numbers here are from the corrected optimiser.