Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2610.12028

Policy Synthesis for Finite Populations of MDP Agents under Aggregate Reach-Avoid Chance Constraints

Jie Fu·Anamika Dubey
View original source ↗Read source document ↗Export metadata ↓

Abstract

Consider a finite population of agents with decoupled Markov transition dynamics and empirical-density feedback, subject to the following constraints: with probability at least $1-δ_r$, at least a fraction $α_r$ of agents must reach a target region at some time $t^*$, while, at each time up to $t^*$, the unsafe population fraction must remain below $β_u$ with probability at least $1-δ_u$. However, standard mean-field methods enforce these constraints only in expectation, which fails to account for stochastic fluctuations at finite fleet size $N$. To address this control problem, we propagate the second-order moment (variance) of the empirical density alongside the mean-field trajectory via a discrete-time Lyapunov recursion, and apply the Cantelli inequality to convert chance constraints into tractable deterministic conditions on the moments of the empirical density. We then incorporate these moment-based surrogate constraints into a gradient-based sequential convex approximation procedure for density-feedback policy synthesis. We further introduce additional moment-error bounds to construct a rigorous finite-$N$ certificate. The method is evaluated on a gridworld environment and a power-system EV-charging aggregation problem and compared with a standard deterministic population-level LP baseline.

Explore related subjects

eess.SYcs.MAcs.SY

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Jie Fu, Anamika Dubey. 2026-10-08. Policy Synthesis for Finite Populations of MDP Agents under Aggregate Reach-Avoid Chance Constraints. https://arxiv.org/abs/2610.12028

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Reinforcement Learning based Traffic Signal Control for Queue Minimization: From Lane-based to Laneless Traffic

Efficient traffic signal control (TSC) is essential for reducing congestion and delays. Traditional TSC methods struggle to capture the dynamics of laneless, heterogeneous traffic. We propose a reinforcement learning (RL) based adaptive TSC framework whose reward depends only on approach-level queue length, motivated by the difficulty of obtaining per-vehicle measurements in laneless traffic, and evaluate it using SUMO on a conventional lane-based, homogeneous (LBHO) intersection and on a non-orthogonal laneless, heterogeneous (LLHE) intersection. On the LBHO site, the lowest mean configuration has about 23\% lower mean queue length than a Dynamic Webster baseline, and the unselected Full-State configuration about 15\% lower. On the LLHE site, all RL configurations have 66-73\% lower mean queue length than a Dynamic Webster baseline under nominal demand, and 63--74\% lower under two higher-demand scenarios evaluated without retraining. Our algorithm also outperforms actuated control and a RESCO-DQN baseline on mean queue length. We additionally release a laneless, heterogeneous-vehicle SUMO framework integrated with RL based signal control, to our knowledge the first such open implementation on a current SUMO version.

eess.SY↗
arXiv2026

A Deep State-Space Model Compression Method using Upper Bound on Output Error

We study deep state-space models (Deep SSMs) that contain linear quadratic-output (LQO) systems as internal blocks and present a compression method with a provable output error guarantee. We first derive an upper bound on the output error between two Deep SSMs and show that the bound can be expressed in terms of the $h^2$-error norms between the layerwise LQO systems. In particular, we show that reducing the $h^2$ approximation errors of the LQO systems placed in shallow layers is effective in reducing the derived upper bound on the output error. Next, we formulate an optimization problem for the derived upper bound and develop a gradient-based MOR method. In the numerical experiments, using the IMDb task from the LRA benchmark, we demonstrate the effectiveness of the proposed upper-bound-based compression method. In particular, we show that the number of trainable parameters can be reduced by approximately 60\% without retraining while maintaining the performance of the original model.

eess.SY↗
arXiv2026

Safe Navigation in Cluttered Environments Via Spline-Based Harmonic Potential Fields

We provide a motion-planning framework for target reaching and obstacle avoidance in cluttered environments. Given a polyhedral decomposition of the feasible space, the agent is guided through a prescribed sequence of cells by a suitable control policy. For each cell, we construct a potential induced by a harmonic function on the unit disk whose Dirichlet boundary condition is represented by a cardinal B-spline curve. A detailed analysis of the curve periodicity, support, and control-point selection provides an explicit analytical construction of these potentials and the associated vector fields. We further characterize inter-cell control continuity and forward invariance of non-exit facets, and introduce a smoothed control law ensuring continuous switching between independently constructed neighboring cells. Numerical results illustrate that the resulting framework safely guides the agent along the prescribed cell sequence from the initial point to the target.

eess.SY↗