Searcharxiv⌕ Search

arXiv subjects

Wangzhi Zhou

Publications and source records attributed to Wangzhi Zhou.

2 recordsLinked to original sources

A Globally Convergent Line-Search Quasi-Newton Method for Discounted Markov Decision Processes

In this paper, we develop a line-search quasi-Newton (LSQN) method for discounted Markov decision processes (MDPs). The method maintains an inverse model to approximate the inverse of the current policy Jacobian and refines it through an inner learning procedure that preserves existing secant conditions and achieves the prescribed accuracy in at most $n$ queries in exact arithmetic for an MDP with $n$ states, even when the initial inverse model is singular. A sign-based direction correction combined with an Armijo line search ensures monotonic decrease of the Bellman residual norm across policy changes without requiring a value-iteration fallback. We prove global Q-linear residual convergence and local Q-superlinear convergence of the value iterates from arbitrary finite initial value vectors and inverse models, without requiring a unique optimal policy. Numerical experiments show competitive performance against nine baselines, while public benchmarks demonstrate that the inner learning procedure typically requires far fewer than $n$ learning queries.

math.OC↗

An early termination strategy for the distributed biased min-consensus protocol under disturbances

The distributed biased min-consensus (DBMC) protocol is an iterative scheme that solves the shortest path problem asymptotically, requiring only local information exchange between neighboring nodes. By appropriately designing the gain function, prior work [1] proposed a DBMC-based system that ensures convergence within a pre-specified time interval. However, this guarantee assumes the absence of disturbances. In this paper, we study the DBMC-based system under disturbances affecting the edge weights. We first establish rigorous error bounds on the resulting state estimates. Building on this analysis, we then propose a practical early termination strategy to prevent potential singularities, specifically, unbounded gain, that may arise in the presence of disturbances, while still ensuring that the shortest paths are correctly identified.Simulations are performed to validate and illustrate the theoretical results.

eess.SY↗