arXiv subjects

Sarah H. Q. Li

Publications and source records attributed to Sarah H. Q. Li.

At least 19 recordsLinked to original sources

arXiv2026

Linear Occupancy Homomorphisms for Discounted Markov Decision Processes

We introduce linear occupancy homomorphisms (LOHs), a class of Markov decision process (MDP) transformations that defines the MDP homomorphism over the feasible occupancy measure space of MDPs. In contrast to classic MDP homomorphism frameworks, which are defined through mappings over the state and state-action spaces, LOHs map between occupancy measure spaces and are representable as linear transformation matrices that expose important structural properties of the homomorphism itself. For this class of MDP transformations, we derive finite dimensional linear inequality constraints that certify when a candidate transformation matrix is a valid LOH and derive the corresponding transformed MDP. We further establish a connection between LOHs, soft MDP homomorphisms, and factored matrix robust MDPs. When the linear transformation matrix is invertible, we prove that the original MDP is equivalent to a constrained MDP after the homomorphic transformation, and derive necessary and sufficient conditions under which the LOH preserves the optimal value. We illustrate the benefits of LOHs in two settings: improving robust MDP solutions by transforming nonrectangular parameter uncertainty sets into homomorphic rectangular MDPs, and decomposing a large MDP into smaller independent MDPs.

math.OC↗

arXiv2026

On the Gap Between H2 Optimal Control and Disturbance Decoupling

We study the relationship between disturbance decoupling (DD) and H2 optimal control for linear time-invariant (LTI) systems, revealing a fundamental gap between DD subspace constraints and semi-definite program (SDP)-based H2 minimization. We show that DD is equivalent to the existence of zero H2 gain without requiring internal stability, whereas SDP-based H2 minimization strictly optimizes over stabilizing controllers and therefore fails to recover DD controllers when the closed-loop dynamics may be marginally stable. Moreover, we show that the trace representation of H2 norms further biases solutions away from complete DD. Motivated by this, we formulate a bilinear matrix inequality (BMI)-constrained optimization program that directly enforces the DD subspace condition to compute DD controllers. We propose a difference-of-convex (DC) iterative algorithm that preserves DD and stability at every iteration, and establish its convergence to Karush-Kuhn-Tucker (KKT) points under standard constraint qualification conditions. Numerical experiments on a four bus power network demonstrate that the proposed algorithm achieves significantly better disturbance rejection while enabling optimization of additional performance metrics. The resulting framework establishes a computationally tractable link between geometric DD theory and optimization-based controller design.

math.OC↗

arXiv2026

Multi-agent Reach-avoid MDP via Potential Games and Low-rank Policy Structure

We optimize finite horizon multi-agent reach-avoid Markov decision process (MDP) via \emph{local feedback policies}. The global feedback policy solution yields global optimality but its communication complexity, memory usage and computation complexity scale exponentially with the number of agents. We mitigate this exponential dependency by restricting the solution space to local feedback policies and show that local feedback policies are rank-one factorizations of global feedback policies, which provides a principled approach to reducing communication complexity and memory usage. Additionally, by demonstrating that multi-agent reach-avoid MDPs over local feedback policies has a potential game structure, we show that iterative best response is a tractable multi-agent learning scheme with guaranteed convergence to deterministic Nash equilibrium, and derive each agent's best response via multiplicative dynamic program (DP) over the joint state space. Numerical simulations across different MDPs and agent sets show that the peak memory usage and offline computation complexity are significantly reduced while the approximation error to the optimal global reach-avoid objective is maintained.

eess.SY↗

arXiv2026

When the Correct Model Fails: The Optimality of Stackelberg Equilibria with Follower Intention Updates

We study a two-player dynamic Stackelberg game where the follower's intention is unknown to the leader. Classical formulations of the Stackelberg equilibrium (SE) assume that the follower's best response (BR) function is known to the leader. However, this is not always true in practice. We study a setting in which the leader receives updated beliefs about the follower BR before the end of the game, such that the update prompts the leader and subsequently the follower to re-optimize their strategies. We characterize the optimality guarantees of the SE solutions under this belief update for both open loop and feedback information structures. Interestingly, we prove that in general, assuming an incorrect follower's BR may lead to a lower leader cost over the entire game than knowing the true follower's BR. We support these results with numerical examples in a linear quadratic (LQ) Stackelberg game, and use Monte Carlo simulations to show that the instances of incorrect BR achieving lower leader costs are non-trivial in collision avoidance LQ Stackelberg games.

eess.SY↗

arXiv2026

Distributionally Robust Tolls for Traffic Networks with Affine Latency Functions

In network congestion games, system operators often utilize latency models, estimated from real-world traffic flow and travel time data, to design monetary incentives which steer equilibrium user behaviors towards lowering system-wide latency. This work studies the impact of latency model uncertainty when designing incentives in non-atomic network congestion games. Our approach leverages distributionally robust optimization (DRO), which captures data-driven uncertainty in latency models by considering worst-case distribution shifts. We prove that, under mild and practically relevant assumptions, the distributionally robust tolling problem in single origin-destination, affine-latency congestion games can be solved via convex programming. Numerical simulations illustrate that tolls designed to be distributionally robust against unknown disturbances can outperform tolls designed using fixed, nominal disturbance models in minimizing system-wide latency.

eess.SY↗

arXiv2026

Language Conditioning Improves Accuracy of Aircraft Goal Prediction in Non-Towered Airspace

Autonomous aircraft must safely operate in non-towered airspace, where coordination relies on voice-based communication among human pilots. Safe operation requires an aircraft to predict the intent, and corresponding goal location, of other aircraft. This paper introduces a multimodal framework for aircraft goal prediction that integrates natural language understanding with spatial reasoning to improve autonomous decision-making in such environments. We leverage automatic speech recognition and large language models to transcribe and interpret pilot radio calls, identify aircraft, and extract discrete intent labels. These intent labels are fused with observed trajectories to condition a temporal convolutional network and Gaussian mixture model for probabilistic goal prediction. Our method significantly reduces goal prediction error compared to baselines that rely solely on motion history, demonstrating that language-conditioned prediction increases prediction accuracy. Experiments on a real-world dataset from a non-towered airport validate the approach and highlight its potential to enable socially aware, language-conditioned robotic motion planning.

cs.RO↗

arXiv2026

Allocating Corrective Control to Mitigate Multi-agent Safety Violations Under Private Preferences

We propose a novel framework that computes the corrective control efforts to ensure joint safety in multi-agent dynamical systems. This framework efficiently distributes the required corrective effort without revealing individual agents' private preferences. Our framework integrates high-order control barrier functions (HOCBFs), which enforce safety constraints with formal guarantees of safety for complex dynamical systems, with a privacy-preserving resource allocation mechanism based on the progressive second price (PSP) auction. When a joint safety constraint is violated, agents iteratively bid on new corrective efforts via 'avoidance credits' rather than explicitly solving for feasible corrective efforts that remove the safety violation. The resulting correction, determined via a second price payment rule, coincides with the socially optimal safe distribution of corrective actions. Critically, the bidding process achieves this optimal allocation efficiently and without revealing private preferences of individual agents. We demonstrate this method through multi-robot hardware experiments on the Robotarium platform.

eess.SY↗

arXiv2025

A Markov Decision Process Framework for Early Maneuver Decisions in Satellite Collision Avoidance

We develop a Markov decision process (MDP) framework to autonomously make guidance decisions for satellite collision avoidance maneuver (CAM) and a reinforcement learning policy gradient (RL-PG) algorithm to enable direct optimization of guidance policy using historic CAM data. In addition to maintaining acceptable collision risks, this approach seeks to minimize the average propellant consumption of CAMs by making early maneuver decisions. We model CAM as a continuous state, discrete action and finite horizon MDP, where the critical decision is determining when to initiate the maneuver. The MDP models decision rewards using analytical models of collision risk, propellant consumption, and transit orbit geometry. By deciding to maneuver earlier than conventional methods, the Markov policy effectively favors CAMs that achieve comparable rates of collision risk reduction while consuming less propellant. Using historical data of tracked conjunction events, we verify this framework and conduct an extensive parameter-sensitivity study. When evaluated on synthetic conjunction events, the trained policy consumes significantly less propellant overall and per maneuver in comparison to a conventional cut-off policy that initiates maneuvers 24 hours before the time of closest approach (TCA). On historical conjunction events, the trained policy consumes more propellant overall but consumes less propellant per maneuver. For both historical and synthetic conjunction events, the trained policy is slightly more conservative in identifying conjunctions events that warrant CAMs in comparison to cutoff policies.

cs.LG↗

arXiv2025

MPC for momentum counter-balanced and zero-impulse contact with a free-spinning satellite

In on-orbit robotics, a servicer satellite's ability to make contact with a free-spinning target satellite is essential to completing most on-orbit servicing (OOS) tasks. This manuscript develops a nonlinear model predictive control (MPC) framework that generates feasible controls for a servicer satellite to achieve zero-impulse contact with a free-spinning target satellite. The overall maneuver requires coordination between two separately actuated modules of the servicer satellite: (1) a moment generation module and (2) a manipulation module. We apply MPC to control both modules by explicitly modeling the cross-coupling dynamics between them. We demonstrate that the MPC controller can enforce actuation and state constraints that prior control approaches could not account for. We evaluate the performance of the MPC controller by simulating zero-impulse contact scenarios with a free-spinning target satellite via numerical Monte Carlo (MC) trials and comparing the simulation results with prior control approaches. Our simulation results validate the effectiveness of the MPC controller in maintaining spin synchronization and zero-impulse contact under operation constraints, moving contact location, and observation and actuation noise.

eess.SY↗

arXiv2024

Computing Optimal Joint Chance Constrained Control Policies

We consider the problem of optimally controlling stochastic, Markovian systems subject to joint chance constraints over a finite-time horizon. For such problems, standard Dynamic Programming is inapplicable due to the time correlation of the joint chance constraints, which calls for non-Markovian, and possibly stochastic, policies. Hence, despite the popularity of this problem, solution approaches capable of providing provably-optimal and easy-to-compute policies are still missing. We fill this gap by augmenting the dynamics via a binary state, allowing us to characterize the optimal policies and develop a Dynamic Programming based solution method.

math.OC↗

arXiv2024

Mitigating Transient Bullwhip Effects Under Imperfect Demand Forecasts

Motivated by how forecast errors exacerbate order fluctuations in supply chains, we leverage robust feedback controller synthesis to characterize, compute, and minimize the worst-case order fluctuation experienced by an individual supply chain vendor. Assuming bounded forecast errors and demand fluctuations, we model forecast error and demand fluctuations as inputs to linear inventory dynamics, and use the $\ell_\infty$ gain to define a transient Bullwhip measure. In contrast to the existing Bullwhip measure, the transient Bullwhip measure explicitly depends on the forecast error. This enables us to separately quantify the transient Bullwhip measure's sensitivity to forecast error and demand fluctuations. To compute the controller that minimizes the worst-case peak gain, we formulate an optimization problem with bilinear matrix inequalities and show that it is equivalent to minimizing a quasi-convex function on a bounded domain. We simulate our model for vendors with non-zero perishable rates and order backlogging rates, and prove that the transient Bullwhip measure can be bounded by a monotonic quasi-convex function whose dependency on the product backlog rate and perishing rate is verified in simulation.

cs.ET↗

arXiv2024

A Coupled Optimization Framework for Correlated Equilibria in Normal-Form Game

In competitive multi-player interactions, simultaneous optimality is a key requirement for establishing strategic equilibria. This property is explicit when the game-theoretic equilibrium is the simultaneously optimal solution of coupled optimization problems. However, no such optimization problems exist for the correlated equilibrium, a strategic equilibrium where the players can correlate their actions. We address the lack of a coupled optimization framework for the correlated equilibrium by introducing an {unnormalized game} -- an extension of normal-form games in which the player strategies are lifted to unnormalized measures over the joint actions. We show that the set of fully mixed generalized Nash equilibria of this unnormalized game is a subset of the correlated equilibrium of the normal-form game. Furthermore, we introduce an entropy regularization to the unnormalized game and prove that the entropy-regularized generalized Nash equilibrium is a sub-optimal correlated equilibrium of the normal form game where the degree of sub-optimality depends on the magnitude of regularization. We prove that the entropy-regularized unnormalized game has a closed-form solution, and empirically verify its computational efficacy at approximating the correlated equilibrium of normal-form games.

cs.GT↗

arXiv2023

Blameless and Optimal Control under Prioritized Safety Constraints

In many resource-limited optimal control problems, multiple constraints may be enforced that are jointly infeasible due to external factors such as subsystem failures, unexpected disturbances, or fuel limitations. In this manuscript, we introduce the concept of blameless optimality to characterize control actions that a) satisfy the highest prioritized and feasible constraints and b) remain optimal with respect to a mission objective. For a general optimal control problem with jointly infeasible constraints, we prove that a single optimization problem cannot find a blamelessly optimal control sequence. Instead, finding blamelessly optimal control actions requires sequentially solving at least two optimal control problems: one to determine the highest priority level of constraints that is feasible and another to determine the optimal control action with respect to these constraints. We apply our results to a rocket landing scenario in which violating at least one safety-induced landing constraint is unavoidable. Leveraging the concept of blameless optimality, we formulate blamelessly optimal controllers that can autonomously prioritize the constraints most critical to a mission.

math.OC↗

arXiv2023

Alpha-Fair Routing in Urban Air Mobility with Risk-Aware Constraints

In the vision of urban air mobility, air transport systems serve the demands of urban communities by routing flight traffic in networks formed by vertiports and flight corridors. We develop a routing algorithm to ensure that the air traffic flow fairly serves the demand of multiple communities subject to stochastic network capacity constraints. This algorithm guarantees that the flight traffic volume allocated to different communities satisfies the \emph{alpha-fairness conditions}, a commonly used family of fairness conditions in resource allocation. It further ensures robust satisfaction of stochastic network capacity constraints by bounding the coherent risk measures of capacity violation. We prove that implementing the proposed algorithm is equivalent to solving a convex optimization problem. We demonstrate the proposed algorithm using a case study based on the city of Austin. Compared with one that maximizes the total served demands, the proposed algorithm promotes even distributions of served demands for different communities.

math.OC↗

arXiv2023

Set-based value operators for non-stationary Markovian environments

This paper analyzes finite state Markov Decision Processes (MDPs) with uncertain parameters in compact sets and re-examines results from robust MDP via set-based fixed point theory. To this end, we generalize the Bellman and policy evaluation operators to contracting operators on the value function space and denote them as \emph{value operators}. We lift these value operators to act on \emph{sets} of value functions and denote them as \emph{set-based value operators}. We prove that the set-based value operators are \emph{contractions} in the space of compact value function sets. Leveraging insights from set theory, we generalize the rectangularity condition in classic robust MDP literature to a containment condition for all value operators, which is weaker and can be applied to a larger set of parameter-uncertain MDPs and contracting operators in dynamic programming. We prove that both the rectangularity condition and the containment condition sufficiently ensure that the set-based value operator's fixed point set contains its own extrema elements. For convex and compact sets of uncertain MDP parameters, we show equivalence between the classic robust value function and the supremum of the fixed point set of the set-based Bellman operator. Under dynamically changing MDP parameters in compact sets, we prove a set convergence result for value iteration, which otherwise may not converge to a single value function. Finally, we derive novel guarantees for probabilistic path-planning problems in planet exploration and stratospheric station-keeping.

cs.LG↗

arXiv2022

Reducing Collision Risk in Multi-Agent Path Planning: Application to Air traffic Management

To minimize collision risks in the multi-agent path planning problem with stochastic transition dynamics, we formulate a Markov decision process congestion game with a multi-linear congestion cost. Players within the game complete individual tasks while minimizing their own collision risks. We show that the set of Nash equilibria coincides with the first-order KKT points of a non-convex optimization problem. Our game is applied to a historical flight plan over France to reduce collision risks between commercial aircraft.

cs.MA↗

arXiv2022

Adaptive Constraint Satisfaction for Markov Decision Process Congestion Games: Application to Transportation Networks

Under the Markov decision process (MDP) congestion game framework, we study the problem of enforcing population distribution constraints on a population of players with stochastic dynamics and coupled congestion costs. Existing research demonstrates that the constraints on the players' population distribution can be satisfied by enforcing tolls. However, computing the minimum toll value for constraint satisfaction requires accurate modeling of the player's congestion costs. Motivated by settings where an accurate congestion cost model is unavailable (e.g. transportation networks), we consider an MDP congestion game with unknown congestion costs. We assume that a constraint-enforcing authority can repeatedly enforce tolls on a population of players who converges to an $ε$-optimal population distribution for any given toll. We then construct a myopic update algorithm to compute the minimum toll value while ensuring that the constraints are satisfied on average. We analyze how the players' sub-optimal responses to tolls impact the rates of convergence towards the minimum toll value and constraint satisfaction. Finally, we construct a congestion game model for Uber drivers in Manhattan, New York City (NYC) using data from the Taxi and Limousine Commission (TLC) to illustrate how to efficiently reduce congestion while minimizing the impact on driver earnings.

cs.GT↗

arXiv2022

Congestion-aware path coordination game with Markov decision process dynamics

Inspired by the path coordination problem arising from robo-taxis, warehouse management, and mixed-vehicle routing problems, we model a group of heterogeneous players responding to stochastic demands as a congestion game under Markov decision process dynamics. Players share a common state-action space but have unique transition dynamics, and each player's unique cost is a {function} of the joint state-action probability distribution. For a class of player cost functions, we formulate the player-specific optimization problem, prove the equivalence between the Nash equilibrium and the solution of a potential minimization problem, and derive dynamic programming approaches to solve the Nash equilibrium. We apply this game to model multi-agent path coordination and introduce congestion-based cost functions that enable players to complete individual tasks while avoiding congestion with their opponents. Finally, we present a learning algorithm for finding the Nash equilibrium that has linear complexity in the number of players. We demonstrate our game model on a multi-robot warehouse \change{path coordination problem}, in which robots autonomously retrieve and deliver packages while avoiding congested paths.

cs.MA↗