Reasoning · ICLR 2026 Co-first author

ActivationReasoning

ActivationReasoning: Logical Reasoning in Latent Activation Spaces

Giving language models an explicit layer of logic over the concepts they represent.

Read the paper Code

Authors

L. Helff, R. Härle, W. Stammer, F. Friedrich, M. Brack, A. Wüst, H. Shindo, P. Schramowski, K. Kersting

THE IDEA, AT A GLANCE

CONCEPTUAL ILLUSTRATION

Detect the premise

Detect A; represent B and C for inference.

Apply logical rules

Apply supplied rules between concepts.

Guide the model

Use inferred propositions to steer activations.

Detect concepts, connect them with logical rules, and use the conclusions to guide the model.

Motivation

Finding a concept inside a language model does not tell us how to reason with it. Sparse features can reveal what is present, but they are usually passive observations.

Method

ActivationReasoning identifies concept representations, maps active concepts to propositions, and applies user-specified logical rules through forward chaining. Inferred propositions can then guide the model’s activations during generation. The logical rules are supplied explicitly, rather than learned by the framework.

Results

Across the evaluated multi-hop reasoning, abstraction, natural-language reasoning, and context-sensitive safety tasks, the framework improved structured reasoning and control. The experiments demonstrate the approach on both Llama and Gemma backbones, with interventions grounded in inspectable concepts and rules.