arXiv subjects

Felix Köster

Publications and source records attributed to Felix Köster.

12 recordsLinked to original sources

arXiv2026

Reconstructing the information processing capacity of physical systems from noisy observations

Driven dynamical systems can compute when their transient states encode complex transformations of past inputs. The information processing capacity (IPC) framework allows for a detailed accounting of these computational properties, however its interpretation in noisy systems has remained incomplete. In this work, we clarify how noise affects the IPC and how one can reconstruct the noiseless IPC. First, we show how to distinguish the dynamics of an unperturbed system from the noise-free component of the stochastic dynamics: The IPC measured for responses averaged over noise realizations is in general not the same as the IPC of the unperturbed system. We explicitly demonstrate that noise can redistribute computational capacity and sometimes even enhance performance on particular tasks, rather than merely degrading a fixed computation. We then introduce covariance reconstruction by orthogonal projection (CROP), which reconstructs the covariance and IPC of the noise-free component directly from noisy observations, without requiring a detailed model of either the system or the noise. At fixed total measurement budget, numerical tests on a classical nonlinear reservoir show that CROP estimates the noise-free IPC more accurately than the standard practice of ensemble averaging over repeated trials. We find the same advantage in a quantum reservoir subject to unavoidable measurement noise. Our results provide a general route to recovering the computational structure of noisy physical systems from finite observations.

nlin.AO↗

arXiv2026

Adaptive Sensing of Continuous Physical Systems for Machine Learning

Physical dynamical systems can be viewed as natural information processors: their systems preserve, transform, and disperse input information. This perspective motivates learning not only from data generated by such systems, but also how to measure them in a way that extracts the most useful information for a given task. We propose a general computing framework for adaptive information extraction from dynamical systems, in which a trainable attention module learns both where to probe the system state and how to combine these measurements to optimize prediction performance. As a concrete instantiation, we implement this idea using a spatiotemporal field governed by a partial differential equation as the underlying dynamics, though the framework applies equally to any system whose state can be sampled. Our results show that adaptive spatial sensing significantly improves prediction accuracy on canonical chaotic benchmarks. This work provides a perspective on attention-enhanced reservoir computing as a special case of a broader paradigm: neural networks as trainable measurement devices for extracting information from physical dynamical systems.

cs.LG↗

arXiv2026

Reservoir Computing as a Language Model

Large Language Models (LLM) have dominated the science and media landscape duo to their impressive performance on processing large chunks of data and produce human-like levels of text. Nevertheless, their huge energy demand and slow processing are still a bottleneck to further increasing quality while also making the models accessible to everyone. To solve this bottleneck, we will investigate how reservoir computing performs on natural text processing, which could enable fast and energy efficient hardware implementations. Studies investigating the use of reservoir computing as a language model remain sparse. In this paper, we compare three distinct approaches for character-level language modeling, two different \emph{reservoir computing} approaches, where only an output layer is trainable, and the well-known \emph{transformer}-based architectures, which fully learn an attention-based sequence representation. We explore the performance, computational cost and prediction accuracy for both paradigms by equally varying the number of trainable parameters for all models. Using a consistent pipeline for all three approaches, we demonstrate that transformers excel in prediction quality, whereas reservoir computers remain highly efficient reducing the training and inference speed. Furthermore, we investigate two types of reservoir computing: a \emph{traditional reservoir} with a static linear readout, and an \emph{attention-enhanced reservoir} that dynamically adapts its output weights via an attention mechanism. Our findings underline how these paradigms scale and offer guidelines to balance resource constraints with performance.

cs.CL↗

arXiv2025

Attention-Enhanced Reservoir Computing as a Multiple Dynamical System Approximator

Reservoir computing has proven effective for tasks such as time-series prediction, particularly in the context of chaotic systems. However, conventional reservoir computing frameworks often face challenges in achieving high prediction accuracy and adapting to diverse dynamical problems due to their reliance on fixed weight structures. A concept of an attention-enhanced reservoir computer has been proposed, which integrates an attention mechanism into the output layer of the reservoir computing model. This addition enables the system to prioritize distinct features dynamically, enhancing adaptability and prediction performance. In this study, we demonstrate the capability of the attention-enhanced reservoir computer to learn and predict multiple chaotic attractors simultaneously with a single set of weights, thus enabling transitions between attractors without explicit retraining. The method is validated using benchmark tasks, including the Lorenz system, Rössler system, Henon map, Duffing oscillator, and Mackey-Glass delay-differential equation. Our results indicate that the attention-enhanced reservoir computer achieves superior prediction accuracy, valid prediction times, and improved representation of spectral and histogram characteristics compared to traditional reservoir computing methods, establishing it as a robust tool for modeling complex dynamical systems.

nlin.CD↗

arXiv2024

Attention-Enhanced Reservoir Computing

Photonic reservoir computing has been successfully utilized in time-series prediction as the need for hardware implementations has increased. Prediction of chaotic time series remains a significant challenge, an area where the conventional reservoir computing framework encounters limitations of prediction accuracy. We introduce an attention mechanism to the reservoir computing model in the output stage. This attention layer is designed to prioritize distinct features and temporal sequences, thereby substantially enhancing the prediction accuracy. Our results show that a photonic reservoir computer enhanced with the attention mechanism exhibits improved prediction capabilities for smaller reservoirs. These advancements highlight the transformative possibilities of reservoir computing for practical applications where accurate prediction of chaotic time series is crucial.

cs.ET↗

arXiv2024

Data-Driven Acceleration of Multi-Physics Simulations

Multi-physics simulations play a crucial role in understanding complex systems. However, their computational demands are often prohibitive due to high dimensionality and complex interactions, such that actual calculations often rely on approximations. To address this, we introduce a data-driven approach to approximate interactions among degrees of freedom of no direct interest and thus significantly reduce computational costs. Focusing on a semiconductor laser as a case study, we demonstrate the superiority of this method over traditional analytical approximations in both accuracy and efficiency. Our approach streamlines simulations, offering promise for complex multi-physics systems, especially for scenarios requiring a large number of individual simulations.

physics.comp-ph↗

arXiv2024

Data-Driven Forecasting of Non-Equilibrium Solid-State Dynamics

We present a data-driven approach to efficiently approximate nonlinear transient dynamics in solid-state systems. Our proposed machine-learning model combines a dimensionality reduction stage with a nonlinear vector autoregression scheme. We report an outstanding time-series forecasting performance combined with an easy to deploy model and an inexpensive training routine. Our results are of great relevance as they have the potential to massively accelerate multi-physics simulation software and thereby guide to future development of solid-state based technologies.

physics.comp-ph↗

arXiv2021

The role of delay-times in delay-based Photonic Reservoir Computing

Delay-based reservoir computing has gained a lot of attention due to the relative simplicity with which this concept can be implemented in hardware. However,there is still an misconception about the relationship between the delay-time and the input clock-cycle which has noticeable consequences for the performance. We review the existing literature on this subject and introduce the concept of delay-based reservoir computing in a manner that demonstrates that there is no predefined relationship between these two times-scales. Further, we discuss ways to improve the computing performance of a reservoir formed by delay-coupled oscillators and show the crucial impact of delay-time tuning in those multi-delay systems.

physics.comp-ph↗

arXiv2021

Master memory function for delay-based reservoir computers with single-variable dynamics

We show that many delay-based reservoir computers considered in the literature can be characterized by a universal master memory function (MMF). Once computed for two independent parameters, this function provides linear memory capacity for any delay-based single-variable reservoir with small inputs. Moreover, we propose an analytical description of the MMF that enables its efficient and fast computation. Our approach can be applied not only to reservoirs governed by known dynamical rules such as Mackey-Glass or Ikeda-like systems but also to reservoirs whose dynamical model is not available. We also present results comparing the performance of the reservoir computer and the memory capacity given by the MMF.

cs.ET↗

arXiv2021

Insight into Delay Based Reservoir Computing via Eigenvalue Analysis

In this paper we give a profound insight into the computation capability of delay-based reservoir computing via an eigenvalue analysis. We concentrate on the task-independent memory capacity to quantify the reservoir performance and compare these with the eigenvalue spectrum of the dynamical system. We show that these two quantities are deeply connected, and thus the reservoir computing performance is predictable by analyzing the small signal response of the reservoir. Our results suggest that any dynamical system used as a reservoir can be analyzed in this way. We apply our method exemplarily to a photonic laser system with feedback and compare the numerically computed recall capabilities with the eigenvalue spectrum. Optimal performance is found for a system with the eigenvalues having real parts close to zero and off-resonant imaginary parts.

cs.LG↗

arXiv2020

Limitations of the recall capabilities in delay based reservoir computing systems

We analyze the memory capacity of a delay based reservoir computer with a Hopf normal form as nonlinearity and numerically compute the linear as well as the higher order recall capabilities. A possible physical realisation could be a laser with external cavity, for which the information is fed via electrical injection. A task independent quantification of the computational capability of the reservoir system is done via a complete orthonormal set of basis functions. Our results suggest that even for constant readout dimension the total memory capacity is dependent on the ratio between the information input period, also called the clock cycle, and the time delay in the system. Optimal performance is found for a time delay about 1.6 times the clock cycle

cs.ET↗

arXiv2020

Deep Time-Delay Reservoir Computing: Dynamics and Memory Capacity

The Deep Time-Delay Reservoir Computing concept utilizes unidirectionally connected systems with time-delays for supervised learning. We present how the dynamical properties of a deep Ikeda-based reservoir are related to its memory capacity (MC) and how that can be used for optimization. In particular, we analyze bifurcations of the corresponding autonomous system and compute conditional Lyapunov exponents, which measure the generalized synchronization between the input and the layer dynamics. We show how the MC is related to the systems distance to bifurcations or magnitude of the conditional Lyapunov exponent. The interplay of different dynamical regimes leads to a adjustable distribution between linear and nonlinear MC. Furthermore, numerical simulations show resonances between clock cycle and delays of the layers in all degrees of the MC. Contrary to MC losses in a single-layer reservoirs, these resonances can boost separate degrees of the MC and can be used, e.g., to design a system with maximum linear MC. Accordingly, we present two configurations that empower either high nonlinear MC or long time linear MC.

nlin.AO↗