Searcharxiv⌕ Search

arXiv subjects

Boli Chen

Publications and source records attributed to Boli Chen.

At least 19 recordsLinked to original sources

Distributed Output-Feedback Control for Linear Multichannel Systems

This paper investigates distributed output-feedback stabilization of linear multichannel systems in which agents exchange state estimates without disclosing local control inputs or feedback gains. Remote control actions are treated as unknown inputs and geometrically decoupled from stable quotient-error dynamics. The proposed framework accommodates nontrivial intersections of local loss subspaces by retaining the coupling between the plant and common estimation errors. A Hurwitz condition on the coupled dynamics, together with a sufficient consensus-gain bound, guarantees exponential closed-loop convergence for every fixed feedback selection in a compact family. For polytopic feedback families, a quotient-preserving parameterization of output injections enables joint synthesis of nominal feedback and observer gains via alternating linear matrix inequalities, followed by a convex semidefinite program certifying a common consensus gain. A numerical example demonstrates the effectiveness of the proposed design.

eess.SY↗

Learning Recommendations under Imperfect Human Compliance with Record-Level Differential Privacy

This paper investigates human-in-the-loop recommendation systems, where effectiveness relies on human compliance and privacy concerns restrict the use of historical response data. The pro posed framework models the implemented action as a random response to the recommendation and formulates the recommendation mechanism as a decision-dependent optimisation problem under an unknown compliance distribution. A batched Gaussian process upper confidence bound algorithm is then adapted to learn from locally perturbed historical actions, providing privacy for individual raw records. The analysis establishes record-level differential privacy and derives high-probability simple regret bounds when the compliance distribution is unknown. These bounds characterise how privacy noise, batch size, and behavioural information affect the guaranteed quality of the learned recommendation. In particular, a known upper bound on the non-compliance rate enables the rec ommendation space to be screened before any private data are queried. Simulations of a fixed-state car-following problem with synthetic historical driver responses demonstrate the effectiveness of the proposed algorithm and verify the theoretical findings.

cs.HC↗

Rollout-Time Intervention: Unlocking Agentic Reinforcement Learning Beyond Policy Limits

Reinforcement learning (RL) has emerged as a key paradigm for improving large language model (LLM) agents, enabling stronger reasoning, planning, and tool-use in interactive environments. However, existing agent RL methods remain fundamentally constrained by an on-policy reasoning bottleneck: when tasks exceed the model's current capability boundary, effective trajectories become extremely sparse and training often collapses. We introduce Rollout-Time Intervention (RTI), a technique designed to elicit successful trajectories beyond the policy's intrinsic exploration capacity. RTI operates by leveraging reward-side evidence (i.e., evaluation rubrics) during rollout to convert erroneous trajectories into successful ones, providing effective learning signals for tasks that the policy rarely solves on its own. Although intervention trajectories are inherently off-policy, we find that they can effectively guide policy optimization when trajectories with excessive distributional shifts are filtered out. Extensive experiments across five benchmarks demonstrate that RTI consistently outperforms RL baselines, enabling the post-trained policy to approach the performance of substantially stronger contemporary LLMs and, more importantly, transcend its original capability boundary. The code is available at https://github.com/Alibaba-NLP/qqr.

cs.AI↗

ArenaFlow: From Trajectory Ranking to Hierarchical Credit Propagation for Open-Ended Agent RL

Reinforcement learning has substantially improved large language model (LLM) agents in verifiable domains, but remains difficult to apply to open-ended agent tasks, where solutions are diverse and reliable scalar rewards are hard to obtain. Recent pairwise evaluation methods alleviate reward discrimination collapse by replacing pointwise scoring with relative preferences. However, they still compress rich comparative feedback into a single trajectory-level reward, obscuring decisive intermediate steps and preventing successful behaviors from being consolidated into reusable skills. We propose ArenaFlow, a hierarchical credit propagation framework for open-ended agent reinforcement learning. ArenaFlow leverages tournament-based relative ranking to derive trajectory-level reward signals. Each comparison is further equipped with structured reflective evaluation, which reveals three types of supervision: pivotal success steps, reusable strategy skills, and usage attribution of retrieved skills. At the step level, ArenaFlow propagates trajectory-level advantages to high-confidence pivotal steps according to tournament survival depth, enabling more targeted optimization of local reasoning behaviors. At the skill level, ArenaFlow estimates skill utility from group-level usage attribution and maintains a global skill memory through utility-aware updating, pruning, and retrieval. The resulting high-utility skills further serve as policy priors for future exploration. Extensive experiments validate ArenaFlow's effectiveness on open-ended agent tasks.

cs.CL↗

ARISE-RL: Agentic Rubric-Grounded Iterative Self-Evolution with Reinforcement Learning

Training open-ended agents via reinforcement learning (RL) is hindered by the lack of verifiable gold answers and scalable rubrics. Moreover, even near the model's capability boundary, long-horizon open-ended agentic tasks often yield brittle and unstable rewards, resulting in weak or noisy rollout contrast that obscures fine-grained optimization signals for group-based policy learning. To address these challenges, we propose ARISE-RL, a novel full-cycle self-evolution framework that couples a task/rubric Generator and a reasoning Solver through rubric-mediated co-evolution. The Generator grounds tool-related rubric criteria in real tool observations and is rewarded for producing valid, intermediate-difficulty tasks aligned with the Solver's evolving capability boundary. The Solver, in turn, learns from fine-grained rubric satisfaction signals through multi-step reasoning and tool use. We further introduce Reward-Gated Self-Evolution Distillation (RG-SED), which selectively distills a memory-augmented variant of the same policy back into itself only when the memory yields empirical reward improvement, thereby reducing distribution mismatch and avoiding blind imitation of noisy guidance. Finally, to support rigorous evaluation, we present ECR-Bench, an expert-calibrated rubric benchmark suite covering single-tool deep research and multi-tool travel planning. Extensive experiments demonstrate that ARISE-RL consistently achieves robust and stable overall state-of-the-art performance across all evaluated benchmarks.

cs.AI↗

SecRespond: Benchmarking AI Agents for Real-World Post-Compromise Incident Response

Large Language Model (LLM) agents are increasingly adopted in real-world security operations with access to host artifacts and command-line interfaces (CLIs), making it critical to thoroughly assess their security capabilities. However, existing cybersecurity benchmarks focus on pre-compromise settings where agents are placed in a clean and idealized environment before an attack occurs. This leaves the post-compromise setting underexplored. To address this gap, we introduce SecRespond, the first benchmark for evaluating LLM agents on the post-compromise incident-response workflow. Given a forensic disk snapshot of a compromised host together with the alerts, vulnerability scans, and baseline checks reported by a host security product, agents are required to produce forensic reports on intrusions, baseline risks, and vulnerability risks, together with a remediation plan. We instantiate this task across 10 cyber ranges, each constructed from a distinct compromised cloud host, spanning 4 entry-point types, 21 ATT&CK techniques, and 5 operating systems. We evaluate 23 frontier LLMs on the OpenCode agent harness. Experimental results show that although current agents can reliably uncover the problems exposed by alerts, they struggle to proactively investigate the disk for silent intrusions and to produce comprehensive, verified remediation plans, with no model achieving complete detection and remediation on any single range. This reveals a fundamental bottleneck in building agents for real-world incident response. The benchmark is publicly available at https://github.com/Alibaba-NLP/qqr/tree/main/data/secrespond.

cs.CR↗

To What Extent Can Inherent Communication Noise Guarantee Privacy in Distributed Cooperative Control?

This paper proposes a differentially private distributed cooperative control scheme for multi-agent systems (MAS). Unlike conventional approaches that actively inject artificial noise for privacy protection, this work investigates whether inherent communication noise can itself serve as a natural privacy mechanism. A physically motivated communication-noise model is developed for mobile MAS by incorporating transmitter perturbation, receiver noise, path-loss attenuation, and log-normal shadowing. The resulting effective noise variance depends on inter-agent state differences, thereby capturing the distance-dependent signal perturbation arising in practice. Based on this model, a distributed finite-horizon Linear Quadratic Regulator (LQR) mechanism is designed to achieve formation tracking while protecting agents' private control preferences. Rather than protecting the full local cost function, the proposed privacy formulation focuses on the ratio of the LQR weighting matrices, which captures the trade-off between tracking accuracy and control effort when the quadratic cost structure is publicly known. A set-theoretic sensitivity analysis shows that this weighting-ratio adjacency formulation yields less conservative privacy bounds than gradient-based protection under the considered addition/removal adjacency relation. Theoretical analysis demonstrates that, under suitable design conditions, the proposed mechanism provides bounded cumulative (ε,δ)-differential privacy guarantees for the weighting ratios over an infinite horizon without artificial noise injection. Meanwhile, the cooperative tracking error is shown to converge almost surely and in mean square to a finite random limit, with its expectation remaining bounded. Numerical examples validate the theoretical results and illustrate the resulting privacy-performance trade-off.

eess.SY↗

ArenaRL: Scaling RL for Open-Ended Agents via Tournament-based Relative Ranking

Reinforcement learning has substantially improved the performance of LLM agents on tasks with verifiable outcomes, but it still struggles on open-ended agent tasks with vast solution spaces (e.g., complex travel planning). Due to the absence of objective ground-truth for these tasks, current RL algorithms largely rely on reward models that assign scalar scores to individual responses. We contend that such pointwise scoring suffers from an inherent discrimination collapse: the reward model struggles to distinguish subtle advantages among different trajectories, resulting in scores within a group being compressed into a narrow range. Consequently, the effective reward signal becomes dominated by noise from the reward model, leading to optimization stagnation. To address this, we propose ArenaRL, a reinforcement learning paradigm that shifts from pointwise scalar scoring to intra-group relative ranking. ArenaRL introduces a process-aware pairwise evaluation mechanism, employing multi-level rubrics to assign fine-grained relative scores to trajectories. Additionally, we construct an intra-group adversarial arena and devise a tournament-based ranking scheme to obtain stable advantage signals. Empirical results confirm that the built seeded single-elimination scheme achieves nearly equivalent advantage estimation accuracy to full pairwise comparisons with O(N^2) complexity, while operating with only O(N) complexity, striking an optimal balance between efficiency and precision. Furthermore, to address the lack of full-cycle benchmarks for open-ended agents, we build Open-Travel and Open-DeepResearch, two high-quality benchmarks featuring a comprehensive pipeline covering SFT, RL training, and multi-dimensional evaluation. Extensive experiments show that ArenaRL substantially outperforms standard RL baselines, enabling LLM agents to generate more robust solutions for complex real-world tasks.

cs.LG↗

Multi-Regional Traffic Control with Travel and Charging Demand Co-Management

Urban traffic management is essential for reducing congestion and supporting sustainable mobility. However, the task is becoming more challenging due to the growing penetration of electric vehicles and their charging demands. This paper presents a regional traffic coordination framework that combines route guidance and charging management to improve traffic network efficiency. Regional traffic dynamics are modeled by the macroscopic fundamental diagram, which allows for the analysis of congestion at the system level. The framework jointly optimizes routes and charging decisions, and it also uses demand management to regulate external inflows into the network. A case study on a 16-region urban network demonstrates the effectiveness of the proposed approach.

eess.SY↗

Distributed State Estimation for Discrete-Time Systems With Unknown Inputs: An Optimization Approach

This paper proposes a novel Distributed Unknown Input Observer (DUIO) framework for state estimation in large-scale systems subject to local unknown inputs. We consider systems where outputs are measured by a network of spatially distributed sensors and inputs are introduced through multiple dispersed channels. In this framework, each local node utilizes only its local input and output measurements to estimate the maximal locally reconstructible state. Subsequently, nodes collaboratively reconstruct the whole system state via a distributed optimization algorithm that fuses these partial estimates. We provide a rigorous analysis showing that the estimation error is bounded, with the error bound explicitly dependent on the number of communication iterations per time step and strongly convexity constant determined by the system parameters. Furthermore, to counteract curvature anisotropy induced by poor conditioned system geometry, we embed a normalization step into the distributed optimization procedure. Simulation results demonstrate the effectiveness of the proposed framework and the performance improvements yielded by the normalization procedure.

eess.SY↗

Distributed State Estimation for Discrete-time LTI Systems: the Design Trilemma and a Novel Framework

With the advancement of IoT technologies and the rapid expansion of cyber-physical systems, there is increasing interest in distributed state estimation, where multiple sensors collaboratively monitor large-scale dynamic systems. Compared with its continuous-time counterpart, a discrete-time distributed observer faces greater challenges, as it cannot exploit high-gain mechanisms or instantaneous communication. Existing approaches depend on three tightly coupled factors: (i) system observability, (ii) communication frequency and dimension of the exchanged information, and (iii) network connectivity. However, the interdependence among these factors remains underexplored. This paper identifies a fundamental trilemma among these factors and introduces a general design framework that balances them through an iterative semidefinite programming approach. As such, the proposed method mitigates the restrictive assumptions present in existing works. The effectiveness and generality of the proposed approach are demonstrated through a simulation example.

eess.SY↗

Distributed Unknown Input Observer Design: A Geometric Approach

We present a geometric approach to designing distributed unknown input observers (DUIOs) for linear time-invariant systems, where measurements are distributed across nodes and each node is influenced by \emph{unknown inputs} through distinct channels. The proposed distributed estimation scheme consists of a network of observers, each tasked with reconstructing the entire system state despite having access only to local input-output signals that are individually insufficient for full state observation. Unlike existing methods that impose stringent rank conditions on the input and output matrices at each node, our approach leverages the $(C,A)$-invariant (conditioned invariant) subspace at each node from a geometric perspective. This enables the design of DUIOs in both continuous- and discrete-time settings under relaxed conditions, for which we establish sufficiency and necessity. The effectiveness of our methodology is demonstrated through extensive simulations, including a practical case study on a power grid system.

eess.SY↗

Can Inherent Communication Noise Guarantee Privacy in Distributed Cooperative Control ?

This paper investigates privacy-preserving distributed cooperative control for multi-agent systems within the framework of differential privacy. In cooperative control, communication noise is inevitable and is usually regarded as a disturbance that impairs coordination. This work revisits such noise as a potential privacy-enhancing factor. A linear quadratic regulator (LQR)-based framework is proposed for agents communicating over noisy channels, \textcolor{black}{where the noise variance depends on the relative state differences between neighbouring agents.} The resulting controller achieves formation while protecting the reference signals from inference attacks. It is analytically proven that the inherent communication noise can guarantee bounded $(ε,δ)$-differential privacy without adding dedicated privacy noise, while the \textcolor{black}{system cooperative tracking error} remains bounded and convergent in both the mean-square and almost-sure sense.

eess.SY↗

Distributionally Robust Equilibria over the Wasserstein Distance for Generalized Nash Game

Generalized Nash equilibrium problem (GNEP) is fundamental for practical applications where multiple self-interested agents work together to make optimal decisions. In this work, we study GNEP with shared distributionally robust chance constraints (DRCCs) for incorporating inevitable uncertainties. The DRCCs are defined over the Wasserstein ball, which can be explicitly characterized even with limited sample data. To determine the equilibrium of the GNEP, we propose an exact approach to transform the original computationally intractable problem into a deterministic formulation using the Nikaido-Isoda function. Specifically, we show that when all agents' objectives are quadratic in their respective variables, the equilibrium can be obtained by solving a typical mixed-integer nonlinear programming (MINLP) problem, where the integer and continuous variables are decoupled in both the objective function and the constraints. This structure significantly improves computational tractability, as demonstrated through a case study on the charging station pricing problem.

eess.SY↗

A Game-Theoretic Predictive Control Framework with Statistical Collision Avoidance Constraints for Autonomous Vehicle Overtaking

This work develops a control framework for the autonomous overtaking of connected and automated vehicles (CAVs) in a mixed traffic environment, where the overtaken vehicle is an unconnected but interactive human-driven vehicle. The proposed method, termed the Game-Theoretic, PRedictive Overtaking (GT-PRO) strategy, successfully decouples the longitudinal and lateral vehicle dynamics of the CAV and comprehensively coordinates these decoupled dynamics via innovative longitudinal and lateral model predictive (MPC) based controllers, respectively. To address the real-time interactive behavior of the human-driven overtaken vehicle, a dynamic Stackelberg game-based bilevel optimization is solved by the lateral controller to directly control the CAV lateral motion and predict the overtaken vehicle longitudinal responses that are subsequently shared with a stochastic MPC that governs the CAV longitudinal motion. The proposed strategy exploits a comprehensive real-world dataset, which captures human driver responses when being overtaken, to tune the game-theoretic lateral controller according to the most common human responses, and to statistically characterize human uncertainties and hence implement a collision avoidance chance constraint for the stochastic longitudinal controller. The simulation results for both polite and aggressive human response case studies of the overtaken vehicle demonstrate that the proposed GT-PRO can achieve for this range of human driver responsiveness, safer, more efficient, and more comfortable autonomous overtaking, as compared to existing autonomous overtaking approaches in the literature. Furthermore, the results suggest that the GT-PRO method is capable of real-time implementation.

eess.SY↗

Distributed Finite-Horizon Optimal Control for Consensus with Differential Privacy Guarantees

This paper addresses the problem of privacy-preserving consensus control for multi-agent systems (MAS) using differential privacy. We propose a novel distributed finite-horizon linear quadratic regulator (LQR) framework, in which agents share individual state information while preserving the confidentiality of their local pairwise weight matrices, which are considered sensitive data in MAS. Protecting these matrices effectively safeguards each agent's private cost function and control preferences. Our solution injects consensus error-dependent Laplace noise into the communicated state information and employs a carefully designed time-dependent scaling factor in the local cost functions. {This approach guarantees bounded consensus and achieves rigorous $ε$-differential privacy for the weight matrices without relying on specific noise distribution assumptions.} Additionally, we analytically characterize the trade-off between consensus accuracy and privacy level, offering clear guidelines on how to enhance consensus performance through appropriate scaling of the LQR weight matrices and the privacy budget.

eess.SY↗

Distributed Unknown Input Observer Design with Relaxed Conditions: Theory and Application to Vehicle Platooning

Designing observers for linear systems with both known and unknown inputs is an important problem in several research contexts, for example, fault diagnosis and fault-tolerant control, and cyber-secure control systems, and presents significant challenges in distributed state estimation due to the limited sensing capabilities of individual nodes. Existing methods typically impose an individual input-to-output rank condition on each estimator node, which severely restricts applicability in practical applications. This paper presents a novel distributed unknown-input observer design scheme based on a geometric approach under much weaker assumptions than the ones available in the literature. By leveraging the properties of the $(C, A)$-invariant (conditioned invariant) subspace at each node, our methodology aims at reconstructing portions of the system state that remain unaffected by local unknown inputs, while integrating these estimates via a network-based information exchange. A case study on vehicle platoon control shows the effectiveness of the proposed approach.

eess.SY↗

Bridging Centralized and Distributed Frameworks in Unknown Input Observer Design

State estimation for linear time-invariant systems with unknown inputs is a fundamental problem in various research domains. In this article, we establish conditions for the design of unknown input observers (UIOs) from a geometric approach perspective. Specifically, we derive a necessary and sufficient geometric condition for the existence of a centralized UIO. Compared to existing results, our condition offers a more general design framework, allowing designers the flexibility to estimate partial information of the system state. Furthermore, we extend the centralized UIO design to distributed settings. In contrast to existing distributed UIO approaches, which require each local node to satisfy the rank condition regarding the unknown input and output matrices, our method accommodates cases where a subset of nodes does not meet this requirement. This relaxation significantly broadens the range of practical applications. Simulation results are provided to demonstrate the effectiveness of the proposed design.

eess.SY↗