Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.19853

Does Policy Mirror Descent Converge in Policy with Non-Decomposable Mirror Maps? A Counterexample

Wenye Li·Ke Wei
View original source ↗Read source document ↗Export metadata ↓

Abstract

Policy mirror descent (PMD) refers to a class of policy optimization methods associated with different mirror maps. For a broad class of decomposable Legendre mirror maps encompassing many standard choices, Li and Wei [2026] show that the PMD policy sequence converges to an optimal policy from any admissible initialization under any positive constant step size, even when the optimal policy is not unique. In this note, we provide a counterexample with assistance from OpenAI's GPT-5.6 model, showing that PMD with a non-decomposable Legendre mirror map need not converge in policy.

Explore related subjects

math.OC

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Wenye Li, Ke Wei. 2026-10-14. Does Policy Mirror Descent Converge in Policy with Non-Decomposable Mirror Maps? A Counterexample. https://arxiv.org/abs/2610.19853

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

A Safe Approximation Based on Mixed-Integer Optimization for Non-Convex Distributional Robustness Governed by Univariate Indicator Functions

In this work, we present an algorithmically tractable safe approximation of distributionally robust optimization (DRO) problems that contain univariate indicator functions. The latter appear in different applications, but render the model nonlinear and nonconvex. The considered ambiguity sets can exploit moment information. Typically, reformulation approaches using duality theory need to make strong assumptions on the structure of the underlying constraints, such as convexity in the decisions or concavity in the uncertainty which cannot be assumed in our setting. We nevertheless present a semi-infinite minorant of the adversarial problem that admits a zero-gap dualization and is subsequently approximated by a discretized counterpart. Under mild assumptions, the latter provides a safe approximation that is a mixed-integer linear problem and can be solved by available standard software. Obtained solutions are guaranteed to be feasible for the original distributionally robust problem. We prove that the approximation of the adversarial problem converges to its true value for increasingly fine discretization, and that the values of the solutions of the safe approximations converge to the value of the true DRO problem under mild assumptions. In addition, the error bound yields an a posteriori enclosure of the true optimal value that can be evaluated in practice. On the practical side, the approach is made concrete for a challenging, fundamental task in material design, namely in particle separation. Computational results for a realistic setting show that the safe approximation meets the imposed quality requirement for every distribution in the ambiguity set, whereas the unprotected solution does not.

math.OC↗
arXiv2026

Geometric Certificates for Deterministic Global Linear Convergence of Overparameterized Projected Gradient Descent

We study projected gradient descent (ProjGD) for the rank-constrained problem $\min\{f(X): \operatorname{rank}(X)\le r\}$ when the unconstrained minimizer $X_*$ has rank $r_*\le r$. Our main contribution is a quantitative geometric certificate relating the normal component of $\nabla f(X)$ to the smallest nonzero singular value of $X$. Under a rank-$(r+r_*)$ restricted curvature condition with constants $0<μ \sqrt{r_*/r}(L-μ)/(2Lμ)$. With an additional rank-$2r$ restricted smoothness assumption, we establish deterministic global linear convergence for $\sqrt{r_*/r}(L-μ)/(2Lμ)<η<1/L$. The result accommodates arbitrary rank overparameterization $r\ge r_*$ and is independent of the singular-value condition number of $X_*$. As a consequence, we obtain an alternative proof of the sharp threshold $δ<(1+\sqrt{r_*/r})^{-1}$ for the absence of spurious local minima in the asymmetric setting, where $δ=(L-μ)/(L+μ)$. This threshold and matching asymmetric examples were established in recent work.

math.OC↗
arXiv2026

Congestion-Aware Charging Coordination for Electric Ride-Hailing Fleets under Stochastic Demand

In this study, we develop a dynamic charging scheduling method that integrates day-ahead charging planning and online reactive vehicle dispatch and charger assignment. The day-ahead planning model minimizes expected charging costs by considering anticipated waiting times and overnight energy replenishment costs. The online model then adapts this plan by maximizing the profit from vehicle dispatching and assigning vehicles to chargers through a reservation-based charging request scheduler to minimize the total charging-related operational time. The developed charging policy is tested on a Manhattan-like study area using synthetic data drawn from NYC yellow taxi data with a fleet size of 100 vehicles given the scenarios of 3000 and 4000 customers/day. The computational results show that our method outperforms different benchmark policies with up to +21.68% profit and +22.22% service rate for 4000 customers relative to the weakest benchmark. Sensitivity analyses are conducted for different system parameters, and managerial insights are discussed.

math.OC↗