Eliza Rezvanova | Searcharxiv

arXiv2025

Continuous time Stochastic optimal control under discrete time partial observations

This work addresses stochastic optimal control problems where the unknown state evolves in continuous time while partial, noisy, and possibly controllable measurements are only available in discrete time. We develop a framework for controlling such systems, focusing on the measure-valued process of the system's state and the control actions that depend on noisy and incomplete data. Our approach uses a stochastic optimal control framework with a probability measure-valued state, which accommodates noisy measurements and integrates them into control decisions through a Bayesian update mechanism. We characterize the control optimality in terms of a sequence of interlaced Hamilton Jacobi Bellman (HJB) equations coupled with controlled impulse steps at the measurement times. For the case of Gaussian-controlled processes, we derive an equivalent HJB equation whose state variable is finite-dimensional, namely the state's mean and covariance. We demonstrate the effectiveness of our methods through numerical examples. These include control under perfect observations, control under no observations, and control under noisy observations. Our numerical results highlight significant differences in the control strategies and their performance, emphasizing the challenges and computational demands of dealing with uncertainty in state observation.

math.OC↗

arXiv2024

Lagrangian Relaxation for Continuous-Time Optimal Control of Coupled Hydrothermal Power Systems Including Storage Capacity and a Cascade of Hydropower Systems with Time Delays

This work considers a short-term, continuous time setting characterized by a coupled power supply system controlled exclusively by a single provider and comprising a cascade of hydropower systems (dams), fossil fuel power stations, and a storage capacity modeled by a single large battery. Cascaded hydropower generators introduce time-delay effects in the state dynamics, which are modeled with differential equations, making it impossible to use classical dynamic programming. We address this issue by introducing a novel Lagrangian relaxation technique over continuous-time constraints, constructing a nearly optimal policy efficiently. This approach yields a convex, nonsmooth optimization dual problem to recover the optimal Lagrangian multipliers, which is numerically solved using a limited memory bundle method. At each step of the dual optimization, we need to solve an optimization subproblem. Given the current values of the Lagrangian multipliers, the time delays are no longer active, and we can solve a corresponding nonlinear Hamilton--Jacobi--Bellman (HJB) Partial Differential Equation (PDE) for the optimization subproblem. The HJB PDE solver provides both the current value of the dual function and its subgradient, and is trivially parallelizable over the state space for each time step. To handle the infinite-dimensional nature of the Lagrange multipliers, we design an adaptive refinement strategy to control the duality gap. Furthermore, we use a penalization technique for the constructed admissible primal solution to smooth the controls while achieving a sufficiently small duality gap. Numerical results based on the Uruguayan power system demonstrate the efficiency of the proposed mathematical models and numerical approach.

math.OC↗