Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.20126

New Modelling and Solution Frameworks for Multiple Objectives, Decomposability and Impulsivity in Continuous-time Markov Decision Processes

Luke Fairley·Rob Shone·Peter Jacko·Jefferson Huang
View original source ↗Read source document ↗Export metadata ↓

Abstract

Impulsive control, in the context of continuous-time Markov decision processes (CTMDPs), refers to actions that instantaneously update the state of the process, leading to theoretical complications. Previous work has overcome these complications, but at the expense of approachability and ease of implementation: those models are mathematically complex and incompatible with pre-existing solution methods. This paper presents an alternative formulation of impulsive control for CTMDPs that is easier to employ in the modelling of real-world problems and can be solved by pre-existing solution methods. This formulation is then extended to allow for multiple objectives and decomposability. A decomposition-based heuristic method is developed that leverages Dantzig-Wolfe decomposition to find approximate Pareto fronts for large multi-objective decomposable impulsive CTMDPs. The modelling framework and solution methodology are applied to a dynamic maintenance problem to demonstrate their computational efficiency, solution space coverage, and solution quality.

Explore related subjects

math.OC

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Luke Fairley, Rob Shone, Peter Jacko, Jefferson Huang. 2026-10-14. New Modelling and Solution Frameworks for Multiple Objectives, Decomposability and Impulsivity in Continuous-time Markov Decision Processes. https://arxiv.org/abs/2610.20126

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

A Safe Approximation Based on Mixed-Integer Optimization for Non-Convex Distributional Robustness Governed by Univariate Indicator Functions

In this work, we present an algorithmically tractable safe approximation of distributionally robust optimization (DRO) problems that contain univariate indicator functions. The latter appear in different applications, but render the model nonlinear and nonconvex. The considered ambiguity sets can exploit moment information. Typically, reformulation approaches using duality theory need to make strong assumptions on the structure of the underlying constraints, such as convexity in the decisions or concavity in the uncertainty which cannot be assumed in our setting. We nevertheless present a semi-infinite minorant of the adversarial problem that admits a zero-gap dualization and is subsequently approximated by a discretized counterpart. Under mild assumptions, the latter provides a safe approximation that is a mixed-integer linear problem and can be solved by available standard software. Obtained solutions are guaranteed to be feasible for the original distributionally robust problem. We prove that the approximation of the adversarial problem converges to its true value for increasingly fine discretization, and that the values of the solutions of the safe approximations converge to the value of the true DRO problem under mild assumptions. In addition, the error bound yields an a posteriori enclosure of the true optimal value that can be evaluated in practice. On the practical side, the approach is made concrete for a challenging, fundamental task in material design, namely in particle separation. Computational results for a realistic setting show that the safe approximation meets the imposed quality requirement for every distribution in the ambiguity set, whereas the unprotected solution does not.

math.OC↗
arXiv2026

Geometric Certificates for Deterministic Global Linear Convergence of Overparameterized Projected Gradient Descent

We study projected gradient descent (ProjGD) for the rank-constrained problem $\min\{f(X): \operatorname{rank}(X)\le r\}$ when the unconstrained minimizer $X_*$ has rank $r_*\le r$. Our main contribution is a quantitative geometric certificate relating the normal component of $\nabla f(X)$ to the smallest nonzero singular value of $X$. Under a rank-$(r+r_*)$ restricted curvature condition with constants $0<μ \sqrt{r_*/r}(L-μ)/(2Lμ)$. With an additional rank-$2r$ restricted smoothness assumption, we establish deterministic global linear convergence for $\sqrt{r_*/r}(L-μ)/(2Lμ)<η<1/L$. The result accommodates arbitrary rank overparameterization $r\ge r_*$ and is independent of the singular-value condition number of $X_*$. As a consequence, we obtain an alternative proof of the sharp threshold $δ<(1+\sqrt{r_*/r})^{-1}$ for the absence of spurious local minima in the asymmetric setting, where $δ=(L-μ)/(L+μ)$. This threshold and matching asymmetric examples were established in recent work.

math.OC↗
arXiv2026

Congestion-Aware Charging Coordination for Electric Ride-Hailing Fleets under Stochastic Demand

In this study, we develop a dynamic charging scheduling method that integrates day-ahead charging planning and online reactive vehicle dispatch and charger assignment. The day-ahead planning model minimizes expected charging costs by considering anticipated waiting times and overnight energy replenishment costs. The online model then adapts this plan by maximizing the profit from vehicle dispatching and assigning vehicles to chargers through a reservation-based charging request scheduler to minimize the total charging-related operational time. The developed charging policy is tested on a Manhattan-like study area using synthetic data drawn from NYC yellow taxi data with a fleet size of 100 vehicles given the scenarios of 3000 and 4000 customers/day. The computational results show that our method outperforms different benchmark policies with up to +21.68% profit and +22.22% service rate for 4000 customers relative to the weakest benchmark. Sensitivity analyses are conducted for different system parameters, and managerial insights are discussed.

math.OC↗