Foundation models · ICLR 2026 · Time Series Workshop Best Paper Award

Finding the Zeitgeist

Finding the Zeitgeist in Time Series Foundation Models

Uncovering how a forecasting model represents temporal patterns and where they occur.

Read the paper

Authors

F. Divo, M. Kraus, R. Härle, P. Ahrend, K. Kersting

THE IDEA, AT A GLANCE

CONCEPTUAL ILLUSTRATION

A frozen forecasting model

Read patch-level hidden states from Chronos-2.

Learn without concept labels

Reconstruct the hidden states using a sparse set of features.

Inspect temporal roles

Find features sensitive to transitions and the endings of patterns.

Learn sparse features from frozen Chronos-2 hidden states, then inspect when they activate. Shaded regions are conceptual examples, not experimental traces.

Motivation

Time-series foundation models can transfer across forecasting tasks, but their internal representations are difficult to interpret. Do they encode simple signal shapes, or richer structure about when a pattern starts, ends, or changes?

Method

We keep Chronos-2 frozen and train BatchTopK sparse autoencoders on synthetic time series to reconstruct its patch-level hidden states without concept labels. We evaluate the learned features using synthetic temporal concepts, then inspect which time-series patches activate individual neurons.

Results

Shallow classifiers recover synthetic temporal concepts from the SAE features. Manual inspection also reveals neurons sensitive to temporal roles: examples activate just after downward transitions or near the end of low-frequency oscillations. The experiments focus on univariate time series and frozen Chronos-2.