Interpretability · NeurIPS 2025 Spotlight
Measuring & guiding monosemanticity
Measuring and Guiding Monosemanticity
Making a latent feature correspond more reliably to a concept we actually care about.
THE IDEA, AT A GLANCE
CONCEPTUAL ILLUSTRATIONMeasure specificity
Check how a concept is localized across features.
Guide the representation
Train with labeled concepts as an additional signal.
Detect & steer
Guide target A while other features can remain entangled.
Motivation
A feature that responds to many unrelated concepts is difficult to interpret and risky to steer. Sparse autoencoders help expose features, but sparsity alone does not guarantee that each feature has one clear meaning.
Method
The Feature Monosemanticity Score measures how well a concept is captured by a single feature and how its representation is distributed across other features. Guided Sparse Autoencoders use concept labels during training to localize target concepts in designated latent dimensions.
Results
On toxicity, writing style, and privacy attributes, guidance improved measured monosemanticity and concept detection. It also enabled more targeted steering with less degradation in generation quality than the compared methods. These findings concern the supervised target concepts, rather than guaranteeing that every learned feature has one meaning.