Reinforcement-Learning-Guided Data-Driven Estimation of Spectral Properties of Stochastic Koopman Semigroups

Abstract

Koopman spectral analysis turns nonlinear stochastic dynamics into a linear evolution of observables and gives access to decay rates, oscillatory modes, and metastable behavior. In practice, however, EDMD, SDMD, and related estimators depend strongly on where the trajectory data are collected. If most trajectories start in regions that carry little spectral information, the leading eigenvalues and eigenfunctions can be poorly estimated even with a rich dictionary. We propose \emph{Reinforced SDMD}, a data-acquisition method that couples Stochastic Dynamic Mode Decomposition with reinforcement learning. The RL agent chooses trajectory-initialization regions, SDMD updates the Koopman approximation, and a spectral-consistency reward evaluates the estimated eigenpairs on the newly generated data. An exploration bonus is added to avoid repeatedly sampling only a small part of the state space. We test multi-armed bandits, DQN, and PPO on stochastic double-well, Duffing, and FitzHugh--Nagumo systems. The learned policies place more samples in regions that are useful for estimating the leading Koopman eigenpairs. We also give an error-propagation analysis showing how SDMD operator error enters the corresponding bandit, approximate value-iteration, and approximate policy-iteration bounds.