arXiv subjects

Xiaoming Duan

Publications and source records attributed to Xiaoming Duan.

At least 19 recordsLinked to original sources

arXiv2026

Reinforcement Learning with Temporal-Logic-Based Causal Diagrams

We study a class of reinforcement learning (RL) tasks where the objective of the agent is to accomplish temporally extended goals. In this setting, a common approach is to represent the tasks as deterministic finite automata (DFA) and integrate them into the state-space for RL algorithms. However, while these machines model the reward function, they often overlook the causal knowledge about the environment. To address this limitation, we propose the Temporal-Logic-based Causal Diagram (TL-CD) in RL, which captures the temporal causal relationships between different properties of the environment. We exploit the TL-CD to devise an RL algorithm in which an agent requires significantly less exploration of the environment. To this end, based on a TL-CD and a task DFA, we identify configurations where the agent can determine the expected rewards early during an exploration. Through a series of case studies, we demonstrate the benefits of using TL-CDs, particularly the faster convergence of the algorithm to an optimal policy due to reduced exploration of the environment.

cs.AI↗

arXiv2026

BridgeFlow: Fast and Robust SE(2)-Equivariant Motion Planning with Flow Matching

In robotic motion planning, equivariance to rigid body transformations is crucial for robust spatial generalization. However, current learning-based planners face a critical dilemma: they either lack inherent equivariance, treating transformed tasks as novel scenarios, or enforce it via computationally expensive specialized architectures that bottleneck real-time inference. To break this trade-off, we propose BridgeFlow, a fast and strictly SE(2)-equivariant generative motion planning framework. Rather than relying on heavy equivariant networks, BridgeFlow achieves exact spatial equivariance via a lightweight task-centric canonicalization module, enabling generalization using standard architectures. To further accelerate inference, we pair a Brownian bridge informative prior with context-aware mini-batch optimal transport. This constructs a straightened vector field that minimizes transport costs and stabilizes training. Furthermore, environmental awareness is explicitly embedded via Classifier-Free Guidance. Evaluations in dense 2D environments and on a 7-DoF Franka manipulator demonstrate that BridgeFlow achieves up to a 15x inference speedup and a 2x higher valid trajectory rate over state-of-the-art diffusion baselines, alongside robust generalization to entirely unseen environments and arbitrary spatial transformations.

cs.RO↗

arXiv2026

Minimizing Worst-Case Weighted Latency for Multi-Robot Persistent Monitoring: Theory and RL-Based Solutions

We study multi-robot persistent monitoring on weighted graphs, where node weights encode monitoring priorities and edge weights encode travel distances. The goal is to design joint robot trajectories that minimize the worst-case weighted latency across all nodes over an infinite time horizon. The widely adopted worst-case latency objective evaluates team performance over the entire time horizon and therefore may fail to distinguish strategies with poor transient behavior but strong asymptotic performance. To address this limitation, we propose a family of tail-performance objectives that generalize the standard objective and study the resulting functional optimization problems. We establish several key theoretical properties, including the existence of optimal strategies, relationships among the proposed objectives and their corresponding optimization problems, approximation by periodic solutions to arbitrary accuracy, and reductions to event-driven decision models with discretized waiting times. Building on these results, we construct an equivalent event-driven Markov decision process (MDP), called the Tail Worst-case Latency-Optimizing Markov Decision Process (TWLO-MDP), which reformulates the tail-performance objective as a standard average-reward criterion. We then develop reinforcement-learning-based solution methods for the TWLO-MDP and introduce the multi-robot monitoring benchmark (M2Bench), a unified platform that supports the evaluation and comparison of heuristic and learning-based monitoring algorithms. Experiments on synthetic and realistic monitoring scenarios show that our methods effectively reduce the worst-case weighted latency and outperform representative baselines.

cs.RO↗

arXiv2026

3DIOC: Direct Data-Driven Inverse Optimal Control for LTI Systems

This paper addresses the Direct Data-Driven Inverse Optimal Control (3DIOC) problem for linear time-invariant (LTI) systems under the linear quadratic (LQ) control. Unlike traditional approaches that require system identification, the proposed method learns the underlying objective function directly from measured input-output trajectories. Leveraging the input-output representation of LTI systems via the Fundamental Lemma, we derive a model-free optimality necessary condition (ONC) for the forward LQ problem, which forms the basis for formulating and solving an inverse optimal control problem. We also provide an identifiability condition to ensure the uniqueness of the inverse solution. While the ONC-based IOC approach is effective in the noise-free case, its performance is not promising when the data is corrupted with noises. We then reformulate the 3DIOC as a bi-level optimization problem, which is solved using iterative gradient descent and offers solution guarantees. Furthermore, we analyze the relationship between the solution sets of the two proposed formulations, providing practical insights into their selection. The simulation results validate the effectiveness and performance of our proposed methods.

math.OC↗

arXiv2026

Multiplayer Reach-Avoid Differential Games with Defender-Side Information Delay

We consider a class of pursuit-evasion games in which multiple defenders and attackers move in the plane with bounded speeds, while each defender observes the states of other agents with a constant time delay. For the one-attacker-one-defender case, we derive an explicit analytical characterization of the attacker's delayed attack region and prove its convexity under mild assumptions. When the defender can guarantee capture, we formulate a convex optimization problem to compute the capture point and derive optimal strategies for both players. These strategies are shown to constitute a subgame-perfect Nash equilibrium by exploiting the sequential structure induced by the information delay. The analysis is further extended to the one-attacker-multiple-defender scenario and to the general multiplayer setting. In the latter case, delay-aware pairwise winning relations are incorporated into a maximum matching formulation to address the defender-attacker assignment. Numerical simulations for one-on-one, one-vs-multiple, and multi-agent cases validate the theoretical results and illustrate the impact of information delay on game outcomes and optimal strategies.

eess.SY↗

arXiv2026

A Communication-Efficient Distributed Optimization Algorithm for Problems with Coupling Constraints

Resource allocation is a fundamental problem in Industrial Internet of Things (IIoT) systems, in which devices work together under limited communication bandwidth to complete diverse tasks. This paper proposes a communication-efficient distributed optimization algorithm tailored for problems with coupled constraints. To tackle coupled constraints, we solve the problem via its dual counterpart, and develop a compressed version. Difference compression and dynamic scaling factors are then introduced to mitigate compression errors. We show that the proposed algorithm converges linearly for strongly convex and smooth objective functions. Numerical simulations validate the theoretical results and demonstrate the efficiency and robustness of the proposed algorithm.

math.OC↗

arXiv2026

Analysis of On-policy Policy Gradient Methods under the Distribution Mismatch

Policy gradient methods are one of the most successful approaches for solving challenging reinforcement learning problems. Despite their empirical successes, many state-of-the-art policy gradient algorithms for discounted problems deviate from the theoretical policy gradient theorem due to the existence of a distribution mismatch. In this work, we analyze the impact of this mismatch on policy gradient methods. Specifically, we first show that in the case of tabular parameterizations, the biased gradient induced by the mismatch still yields a valid first-order characterization of global optimality. Then, we extend this analysis to more general parameterizations by deriving explicit bounds on both the state distribution mismatch and the resulting gradient mismatch in episodic and continuing MDPs, which are shown to vanish at least linearly as the discount factor approaches one. Building on these bounds, we further establish guarantees for the biased policy gradient iterates, showing that they approach approximate stationary points with respect to the exact gradient, with asymptotic residuals depending on the discount factor. Our findings offer insights into the robustness of policy gradient methods as well as the gap between theoretical foundations and practical implementations.

cs.LG↗

arXiv2025

Optimal Unpredictable Control for Linear Systems

In this paper, we investigate how to achieve the unpredictability against malicious inferences for linear systems. The key idea is to add stochastic control inputs, named as unpredictable control, to make the outputs irregular. The future outputs thus become unpredictable and the performance of inferences is degraded. The major challenges lie in: i) how to formulate optimization problems to obtain an optimal distribution of stochastic input, under unknown prediction accuracy of the adversary; and ii) how to achieve the trade-off between the unpredictability and control performance. We first utilize both variance and confidence probability of prediction error to quantify unpredictability, then formulate two two-stage stochastic optimization problems, respectively. Under variance metric, the analytic optimal distribution of control input is provided. With probability metric, it is a non-convex optimization problem, thus we present a novel numerical method and convert the problem into a solvable linear optimization problem. Last, we quantify the control performance under unpredictable control, and accordingly design the unpredictable LQR and cooperative control. Simulations demonstrate the unpredictability of our control algorithm. The obtained optimal distribution outperforms Gaussian and Laplace distributions commonly used in differential privacy under proposed metrics.

eess.SY↗

arXiv2025

Pursuit-Evasion Between a Velocity-Constrained Double-Integrator Pursuer and a Single-Integrator Evader

We study a pursuit-evasion game between a double integrator-driven pursuer with bounded velocity and bounded acceleration and a single integrator-driven evader with bounded velocity in a two-dimensional plane. The pursuer's goal is to capture the evader in the shortest time, while the evader attempts to delay the capture. We analyze two scenarios based on whether the capture can happen before the pursuer's speed reaches its maximum. For the case when the pursuer can capture the evader before its speed reaches its maximum, we use geometric methods to obtain the strategies for the pursuer and the evader. For the case when the pursuer cannot capture the evader before its speed reaches its maximum, we use numerical methods to obtain the strategies for the pursuer and the evader. In both cases, we demonstrate that the proposed strategies are optimal in the sense of Nash equilibrium through the Hamilton-Jacobi-Isaacs equation, and the pursuer can capture the evader as long as as its maximum speed is larger than that of the evader. Simulation experiments illustrate the effectiveness of the strategies.

eess.SY↗

arXiv2025

Aucamp: An Underwater Camera-Based Multi-Robot Platform with Low-Cost, Distributed, and Robust Localization

This paper introduces an underwater multi-robot platform, named Aucamp, characterized by cost-effective monocular-camera-based sensing, distributed protocol and robust orientation control for localization. We utilize the clarity feature to measure the distance, present the monocular imaging model, and estimate the position of the target object. We achieve global positioning in our platform by designing a distributed update protocol. The distributed algorithm enables the perception process to simultaneously cover a broader range, and greatly improves the accuracy and robustness of the positioning. Moreover, the explicit dynamics model of the robot in our platform is obtained, based on which, we propose a robust orientation control framework. The control system ensures that the platform maintains a balanced posture for each robot, thereby ensuring the stability of the localization system. The platform can swiftly recover from an forced unstable state to a stable horizontal posture. Additionally, we conduct extensive experiments and application scenarios to evaluate the performance of our platform. The proposed new platform may provide support for extensive marine exploration by underwater sensor networks.

cs.RO↗

arXiv2025

IDEAL: Data Equilibrium Adaptation for Multi-Capability Language Model Alignment

Large Language Models (LLMs) have achieved impressive performance through Supervised Fine-tuning (SFT) on diverse instructional datasets. When training on multiple capabilities simultaneously, the mixture training dataset, governed by volumes of data from different domains, is a critical factor that directly impacts the final model's performance. Unlike many studies that focus on enhancing the quality of training datasets through data selection methods, few works explore the intricate relationship between the compositional quantity of mixture training datasets and the emergent capabilities of LLMs. Given the availability of a high-quality multi-domain training dataset, understanding the impact of data from each domain on the model's overall capabilities is crucial for preparing SFT data and training a well-balanced model that performs effectively across diverse domains. In this work, we introduce IDEAL, an innovative data equilibrium adaptation framework designed to effectively optimize volumes of data from different domains within mixture SFT datasets, thereby enhancing the model's alignment and performance across multiple capabilities. IDEAL employs a gradient-based approach to iteratively refine the training data distribution, dynamically adjusting the volumes of domain-specific data based on their impact on downstream task performance. By leveraging this adaptive mechanism, IDEAL ensures a balanced dataset composition, enabling the model to achieve robust generalization and consistent proficiency across diverse tasks. Experiments across different capabilities demonstrate that IDEAL outperforms conventional uniform data allocation strategies, achieving a comprehensive improvement of approximately 7% in multi-task evaluation scores.

cs.AI↗

arXiv2025

Formation Maneuver Control Based on the Augmented Laplacian Method

This paper proposes a novel formation maneuver control method for both 2-D and 3-D space, which enables the formation to translate, scale, and rotate with arbitrary orientation. The core innovation is the novel design of weights in the proposed augmented Laplacian matrix. Instead of using scalars, we represent weights as matrices, which are designed based on a specified rotation axis and allow the formation to perform rotation in 3-D space. To further improve the flexibility and scalability of the formation, the rotational axis adjustment approach and dynamic agent reconfiguration method are developed, allowing formations to rotate around arbitrary axes in 3-D space and new agents to join the formation. Theoretical analysis is provided to show that the proposed approach preserves the original configuration of the formation. The proposed method maintains the advantages of the complex Laplacian-based method, including reduced neighbor requirements and no reliance on generic or convex nominal configurations, while achieving arbitrary orientation rotations via a more simplified implementation. Simulations in both 2-D and 3-D space validate the effectiveness of the proposed method.

eess.SY↗

arXiv2025

Stochastic Trajectory Optimization for Robotic Skill Acquisition From a Suboptimal Demonstration

Learning from Demonstration (LfD) has emerged as a crucial method for robots to acquire new skills. However, when given suboptimal task trajectory demonstrations with shape characteristics reflecting human preferences but subpar dynamic attributes such as slow motion, robots not only need to mimic the behaviors but also optimize the dynamic performance. In this work, we leverage optimization-based methods to search for a superior-performing trajectory whose shape is similar to that of the demonstrated trajectory. Specifically, we use Dynamic Time Warping (DTW) to quantify the difference between two trajectories and combine it with additional performance metrics, such as collision cost, to construct the cost function. Moreover, we develop a multi-policy version of the Stochastic Trajectory Optimization for Motion Planning (STOMP), called MSTOMP, which is more stable and robust to parameter changes. To deal with the jitter in the demonstrated trajectory, we further utilize the gain-controlling method in the frequency domain to denoise the demonstration and propose a computationally more efficient metric, called Mean Square Error in the Spectrum (MSES), that measures the trajectories' differences in the frequency domain. We also theoretically highlight the connections between the time domain and the frequency domain methods. Finally, we verify our method in both simulation experiments and real-world experiments, showcasing its improved optimization performance and stability compared to existing methods.

cs.RO↗

arXiv2025

Model Selection for Inverse Reinforcement Learning via Structural Risk Minimization

Inverse reinforcement learning (IRL) usually assumes the reward function model is pre-specified as a weighted sum of features and estimates the weighting parameters only. However, how to select features and determine a proper reward model is nontrivial and experience-dependent. A simplistic model is less likely to contain the ideal reward function, while a model with high complexity leads to substantial computation cost and potential overfitting. This paper addresses this trade-off in the model selection for IRL problems by introducing the structural risk minimization (SRM) framework from statistical learning. SRM selects an optimal reward function class from a hypothesis set minimizing both estimation error and model complexity. To formulate an SRM scheme for IRL, we estimate the policy gradient from given demonstration as the empirical risk, and establish the upper bound of Rademacher complexity as the model penalty of hypothesis function classes. The SRM learning guarantee is further presented. In particular, we provide the explicit form for the linear weighted sum setting. Simulations demonstrate the performance and efficiency of our algorithm.

cs.LG↗

arXiv2024

Pursuit Winning Strategies for Reach-Avoid Games with Polygonal Obstacles

This paper studies a multiplayer reach-avoid differential game in the presence of general polygonal obstacles that block the players' motions. The pursuers cooperate to protect a convex region from the evaders who try to reach the region. We propose a multiplayer onsite and close-to-goal (MOCG) pursuit strategy that can tell and achieve an increasing lower bound on the number of guaranteed defeated evaders. This pursuit strategy fuses the subgame outcomes for multiple pursuers against one evader with hierarchical optimal task allocation in the receding-horizon manner. To determine the qualitative subgame outcomes that who is the game winner, we construct three pursuit winning regions and strategies under which the pursuers guarantee to win against the evader, regardless of the unknown evader strategy. First, we utilize the expanded Apollonius circles and propose the onsite pursuit winning that achieves the capture in finite time. Second, we introduce convex goal-covering polygons (GCPs) and propose the close-to-goal pursuit winning for the pursuers whose visibility region contains the whole protected region, and the goal-visible property will be preserved afterwards. Third, we employ Euclidean shortest paths (ESPs) and construct a pursuit winning region and strategy for the non-goal-visible pursuers, where the pursuers are firstly steered to positions with goal visibility along ESPs. In each horizon, the hierarchical optimal task allocation maximizes the number of defeated evaders and consists of four sequential matchings: capture, enhanced, non-dominated and closest matchings. Numerical examples are presented to illustrate the results.

eess.SY↗

arXiv2024

HiCRISP: An LLM-based Hierarchical Closed-Loop Robotic Intelligent Self-Correction Planner

The integration of Large Language Models (LLMs) into robotics has revolutionized human-robot interactions and autonomous task planning. However, these systems are often unable to self-correct during the task execution, which hinders their adaptability in dynamic real-world environments. To address this issue, we present a Hierarchical Closed-loop Robotic Intelligent Self-correction Planner (HiCRISP), an innovative framework that enables robots to correct errors within individual steps during the task execution. HiCRISP actively monitors and adapts the task execution process, addressing both high-level planning and low-level action errors. Extensive benchmark experiments, encompassing virtual and real-world scenarios, showcase HiCRISP's exceptional performance, positioning it as a promising solution for robotic task planning with LLMs.

cs.RO↗

arXiv2024

A Stochastic Surveillance Stackelberg Game: Co-Optimizing Defense Placement and Patrol Strategy

Stochastic patrol routing is known to be advantageous in adversarial settings; however, the optimal choice of stochastic routing strategy is dependent on a model of the adversary. We adopt a worst-case omniscient adversary model from the literature and extend the formulation to accommodate heterogeneous defenses at the various nodes of the graph. Introducing this heterogeneity leads to interesting new patrol strategies. We identify efficient methods for computing these strategies in certain classes of graphs. We assess the effectiveness of these strategies via comparison to an upper bound on the value of the game. Finally, we leverage the heterogeneous defense formulation to develop novel defense placement algorithms that complement the patrol strategies.

eess.SY↗

arXiv2023

Observation-based Optimal Control Law Learning with LQR Reconstruction

Designing controllers to generate various trajectories has been studied for years, while recently, recovering an optimal controller from trajectories receives increasing attention. In this paper, we reveal that the inherent linear quadratic regulator (LQR) problem of a moving agent can be reconstructed based on its trajectory observations only, which enables one to learn the optimal control law of the agent autonomously. Specifically, the reconstruction of the optimization problem requires estimation of three unknown parameters including the target state, weighting matrices in the objective function and the control horizon. Our algorithm considers two types of objective function settings and identifies the weighting matrices with proposed novel inverse optimal control methods, providing the well-posedness and identifiability proof. We obtain the optimal estimate of the control horizon using binary search and finally reconstruct the LQR problem with above estimates. The strength of learning control law with optimization problem recovery lies in less computation consumption and strong generalization ability. We apply our algorithm to the future control input prediction and the discrepancy loss is further derived. Numerical simulations and hardware experiments on a self-designed robot platform illustrate the effectiveness of our work.

eess.SY↗