Searcharxiv⌕ Search

arXiv subjects

Anif N. Shikder

Publications and source records attributed to Anif N. Shikder.

3 recordsLinked to original sources

Symmetry in Time: A Representation Theory of Linear Recurrent Memory

State space models (SSMs) are an emerging alternative to Transformers. SSMs offer linear-time inference and parallel training, but most implementations make the parallel scan efficient by making the recurrence diagonal: each channel of the hidden state is multiplied by an input-dependent real or complex number. These numbers commute, so the state records which inputs occurred but not their order. We formalize this with representation theory, which explains and fixes the state-tracking failures previously reported for SSMs. The state update defines a linear representation of the monoid of input sequences, and we show that an SSM tracks a group at every sequence length exactly when that group is a quotient of the group generated by its transition matrices. Two consequences follow. Diagonal real- and complex-valued SSMs can track only commutative groups, while the transition matrices of any SSM that can track a group must split into independent blocks over the real, complex, and quaternion numbers. We then introduce HyperSSM, which adds quaternion channels while keeping linear-time inference and parallel training. Trained only on final answers, HyperSSM solves the group tasks of recent state-tracking benchmarks exactly with at most five state numbers. This includes $S_5$ with 32 generators, where previous SSMs are at chance, and HyperSSM generalizes to sequences 25 times longer than those seen in training. Because the theory says what the transitions must encode, the learned weights of the linear recurrence can be interpreted directly. We demonstrate this by recovering the calibration of a real gyroscope, the rotations of imperfect qubit gates and the geometry of protein backbones.

cs.LG↗

Exact expression for maximum Lyapunov exponent during transients in computationally powerful dynamical networks

We study a network whose rich spatiotemporal dynamics have recently been shown to enable dynamics-based computation, including logic gates, short-term memory, and simple encryption. The network's time dynamics can be exactly solved through a nonlinear coordinate transformation. Here, we derive an exact analytical expression for the network's time-dependent maximum Lyapunov exponent (MLE). We demonstrate, both numerically and analytically, that the network exhibits positive MLEs during the transients that are useful for computation. Our framework enables algebraic manipulation of transient lifetimes through network connectivity and initial conditions, providing a rigorous theoretical foundation for understanding and controlling computation with transients.

nlin.CD↗

An explicit operator explains end-to-end computation in the modern neural networks used for sequence and language modeling

We establish a mathematical correspondence between state space models, a state-of-the-art architecture for capturing long-range dependencies in data, and an exactly solvable nonlinear oscillator network. As a specific example of this general correspondence, we analyze the diagonal linear time-invariant implementation of the Structured State Space Sequence model (S4). The correspondence embeds S4D, a specific implementation of S4, into a ring network topology, in which recent inputs are encoded, as waves of activity traveling over the one-dimensional spatial layout of the network. We then derive an exact operator expression for the full forward pass of S4D, yielding an analytical characterization of its complete input-output map. This expression reveals that the nonlinear decoder in the system induces interactions between these information-carrying waves that enable classifying real-world sequences. These results generalize across modern SSM architectures, and show that they admit an exact mathematical description with a clear physical interpretation. These insights enable a new level of interpretability for these systems in terms of nonlinear oscillator networks.

cs.NE↗