arXiv · 2611.14088
Discrete-Time Stochastic Control Beyond i.i.d. Noise
Abstract
We study discrete-time stochastic control problems where the noise driving the state dynamics is a stationary infinite-order chain rather than an i.i.d. sequence. We formulate the problem as a controlled Markov process by enlarging the state space to include the posterior distribution of the noise given the state-action history. Assuming that the sensitivity of the noise transition kernel to past noise values decays exponentially, we show that the expected total variation distance between the full-history and finite-memory noise filters decays at rate $\sqrt{N}ρ^{\sqrt{N}}$. We use this result to construct finite-memory control models and establish explicit near-optimality bounds for their policies. Finally, under suitable ergodicity conditions, we show that Q-learning converges to the optimal solution of the approximate model, so the same performance bounds apply to the learned policies.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Reyajul Hasan Leon, Ali Devran Kara. 2026-11-10. Discrete-Time Stochastic Control Beyond i.i.d. Noise. https://searcharxiv.com/abs/2611.14088
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.