arXiv subjects

Jianglin Qiao

Publications and source records attributed to Jianglin Qiao.

4 recordsLinked to original sources

arXiv2026

RTPO: Reverse-Turn Policy Optimization for Stabilizing Agentic RL Training

Training multi-turn agentic workflows with reinforcement learning (RL) enables large language models to perform complex reasoning, use external tools, and conduct iterative search beyond single-turn settings. Yet multi-turn RL training remains highly unstable, often causing severe performance degradation as the number of turns increases. Through theoretical analysis, we identify three tightly coupled sources of instability: rollout-training context mismatch, weak turn-level credit assignment under sparse terminal rewards, and asynchronous policy drift when short and long trajectories are optimized under different policy versions. We show that these issues share a common structural origin in flattened trajectory optimization and address them through a unified reverse-turn formulation. We propose Reverse-Turn Policy Optimization (RTPO), which organizes multi-turn rollouts as sparse reverse trees and performs turn-level policy updates in temporal reverse order, aligning each decision with its downstream continuation. RTPO enables causally consistent turn-level credit assignment and on-policy continuation to control asynchronous drift. We provide theoretical guarantees showing that RTPO eliminates context mismatch and asynchronous drift under the proposed turn-level formulation, reduces credit bias, and converges to recursive optimality. Experiments on multi-turn agentic RL benchmarks show that RTPO improves upon trajectory- and turn-level baselines by 21.50% and 10.76%, respectively, highlighting its potential to support more stable training for tool-using agents.

cs.AI↗

arXiv2026

Toward Adaptable Multi-Agent Reinforcement Learning: An Assumption-Aware Review

Multi-Agent Reinforcement Learning (MARL) has achieved strong performance in simulated benchmarks, yet real deployments often violate the assumptions under which algorithms are designed and evaluated. Agent populations may change, objectives may shift, centralized information may be unavailable, execution may become asynchronous, and partner policies may be unfamiliar. Existing surveys discuss related desiderata such as scalability, robustness, generalization, and transferability, but these terms often refer to different objects of analysis and different kinds of distributional or structural shift. This survey proposes \textit{adaptability} as an assumption-aware taxonomy for organizing these shifts, rather than as a universal requirement that every MARL algorithm should succeed in every setting. We distinguish three dimensions: \textit{learning adaptability}, which concerns the applicability of learning paradigms under changed training or system assumptions; \textit{policy adaptability}, which concerns the reuse or adaptation of learned policies under deployment-time changes; and \textit{scenario-driven adaptability}, which concerns whether benchmarks and evaluation protocols expose controlled, diagnostically useful shifts. By separating what changes, when the change occurs, what adaptation is allowed, and what success means, the framework clarifies how established concepts fit together and identifies where current MARL evaluation remains underspecified.

cs.AI↗

arXiv2025

Nucleolus Credit Assignment for Effective Coalitions in Multi-agent Reinforcement Learning

In cooperative multi-agent reinforcement learning (MARL), agents typically form a single grand coalition based on credit assignment to tackle a composite task, often resulting in suboptimal performance. This paper proposed a nucleolus-based credit assignment grounded in cooperative game theory, enabling the autonomous partitioning of agents into multiple small coalitions that can effectively identify and complete subtasks within a larger composite task. Specifically, our designed nucleolus Q-learning could assign fair credits to each agent, and the nucleolus Q-operator provides theoretical guarantees with interpretability for both learning convergence and the stability of the formed small coalitions. Through experiments on Predator-Prey and StarCraft scenarios across varying difficulty levels, our approach demonstrated the emergence of multiple effective coalitions during MARL training, leading to faster learning and superior performance in terms of win rate and cumulative rewards especially in hard and super-hard environments, compared to four baseline methods. Our nucleolus-based credit assignment showed the promise for complex composite tasks requiring effective subteams of agents.

cs.MA↗

arXiv2025

Task Allocation in Customer-led Two-sided Markets with Satellite Constellation Services

Multi-agent systems (MAS) are increasingly applied to complex task allocation in two-sided markets, where agents such as companies and customers interact dynamically. Traditional company-led Stackelberg game models, where companies set service prices, and customers respond, struggle to accommodate diverse and personalised customer demands in emerging markets like crowdsourcing. This paper proposes a customer-led Stackelberg game model for cost-efficient task allocation, where customers initiate tasks as leaders, and companies create their strategies as followers to meet these demands. We prove the existence of Nash Equilibrium for the follower game and Stackelberg Equilibrium for the leader game while discussing their uniqueness under specific conditions, ensuring cost-efficient task allocation and improved market performance. Using the satellite constellation services market as a real-world case, experimental results show a 23% reduction in customer payments and a 6.7-fold increase in company revenues, demonstrating the model's effectiveness in emerging markets.

cs.GT↗