arXiv subjects

Wenqiang Zhu

Publications and source records attributed to Wenqiang Zhu.

14 recordsLinked to original sources

arXiv2026

Reinforcement learning with reputation-based adaptive exploration promotes cooperation

Reinforcement learning provides a framework for studying how individuals adjust their behavior through repeated interaction and feedback in social dilemmas. In Q-learning, exploration controls how often agents choose actions other than those favored by their current learned Q-values. Yet existing models usually treat the exploration rate as a constant parameter. In systems with social evaluation, however, trial-and-error behavior carries different costs and opportunities for agents with different reputations, making exploration dependent on social standing rather than uniform across agents. Herein, we develop a spatial prisoner's dilemma model in which Q-learning agents adapt their exploration rates according to local reputation differences, while reputation is updated through an asymmetric, state-dependent rule. Results show that adaptive exploration and asymmetric reputation updating each promote cooperation, but their combination produces a stronger reinforcing effect than either mechanism alone. Low-reputation agents explore more and can recover reputation through cooperation, while high-reputation agents explore less and avoid reputation losses caused by defection. This mechanism also reorganizes cooperation in space, producing a stable checkerboard-like coexistence at intermediate reputation concern. In addition, cooperation is most vulnerable at intermediate baseline exploration rates, whereas stronger asymmetric reputation updating mitigates this exploration-induced disruption. These results suggest that reputation can act not only as a record of past behavior, but also as a dynamic signal that regulates exploratory behavior during learning and thereby stabilizes cooperation.

physics.comp-ph↗

arXiv2026

Stereotyping by strategy standing diversifies cooperation patterns in indirect reciprocity

Indirect reciprocity explains how cooperation evolves through social reputations. People observe others, assign reputations, and condition their future actions on these assignments. This process is cognitively demanding, and stereotyping offers a simpler alternative by replacing individual-level reputation with group-level information. Theoretical models commonly implement stereotyping through exogenously given group labels. In real societies, however, group-level impressions may be associated with observable patterns of behavior. Here we propose a framework of stereotyping by strategy standing, in which mutants may condition their actions on the overall reputation level associated with a resident strategy rather than on the recipient's reputation. We show that this form of stereotyping can diversify stable cooperation in indirect reciprocity. As the strength of stereotyping increases, additional cooperative evolutionarily stable norm-strategy (ESS) pairs emerge in substantial numbers. In particular, we identify eight highly cooperative ESS pairs that become stable under very weak stereotyping. These pairs, which we call the counterparts of the leading eight, share the same social norms as the classical leading eight and differ only in how they prescribe behavior between bad individuals. They are unstable without stereotyping because they can be invaded by their corresponding leading strategies, but they become stable once stereotyping exceeds a critical threshold. Our results suggest that group-level impressions based on strategy standing can provide a coarse-grained informational route to stable cooperation and offer a more behaviorally grounded perspective on how stereotyping affects indirect reciprocity.

physics.soc-ph↗

arXiv2026

The survival of the weakest in a biased donation game

Cooperating first then mimicking the partner's act has been proven to be effective in utilizing reciprocity in social dilemmas. However, the extent to which this, called Tit-for-Tat strategy, should be regarded as equivalent to unconditional cooperators remains controversial. Here, we introduce a biased Tit-for-Tat (T) strategy that cooperates differently toward unconditional cooperators (C) and fellow T players through independent bias parameters. The results show that, even under strong dilemmas in the donation game framework, this three-strategy system can exhibit diverse phase diagrams on the parameter plane. In particular, when T-bias is small and C-bias is large, a ``hidden T phase'' emerges, in which the weakest T strategy dominates. The dominance of the weakened T strategy originates from a counterintuitive mechanism characterizing non-transitive ecological systems: T suppresses its relative fitness to C, rapidly eliminates the cyclic dominance clusters, and subsequently expands slowly to take over the entire population. Analysis in well-mixed populations confirms that this phenomenon arises from structured populations. Our study thus reveals the subtle role of bias regulation in cooperative modes by emphasizing the ``survival of the weakest'' effect in a broader context.

cs.GT↗

arXiv2026

Nonlinear Public Goods Game in Dynamical Environments

The evolutionary mechanisms of cooperative behavior represent a fundamental topic in complex systems and evolutionary dynamics. Real-world collective interactions, particularly in multi-agent systems, are often characterized by behavior-dependent mechanism switching where the environmental state is endogenously shaped by group strategies. However, existing models typically treat such environmental variations as static stochasticity and neglect the closed-loop feedback between environmental states and cooperative behaviors. Here, we introduce a dynamic environmental feedback mechanism into a nonlinear public goods game framework to establish a coevolutionary model that couples environmental states and individual cooperative strategies. Our results demonstrate that the interplay among environmental feedback, nonlinear effects, and environmental randomness can drive the system toward a wide variety of steady-state structures, including full defection, full cooperation, stable coexistence, and periodic limit cycles. Further analysis reveals that asymmetric nonlinear parameters and environmental feedback rates exert significant regulatory effects on cooperation levels and system dynamics. This study not only enriches the theoretical framework of evolutionary game theory but also provides a foundation for modeling environmental feedback loops in scenarios ranging from ecological management to the design of cooperative mechanisms in autonomous systems.

nlin.AO↗

arXiv2026

Indirect Reciprocity with Environmental Feedback

Indirect reciprocity maintains cooperation in stranger societies by mapping individual behaviors onto reputation signals via social norms. Existing theoretical frameworks assume static environments with constant resources and fixed payoff structures. However, in real-world systems, individuals' strategic behaviors not only shape their reputation but also induce collective-level resource changes in ecological, economic, or other external environments, which in turn reshape the incentives governing future individual actions. To overcome this limitation, we establish a co-evolutionary framework that couples moral assessment, strategy updating, and environmental dynamics, allowing the payoff structure to dynamically adjust in response to the ecological consequences of collective actions. We find that this environmental feedback mechanism helps lower the threshold for the emergence of cooperation, enabling the system to spontaneously transition from a low-cooperation state to a stable high-cooperation regime, thereby reducing the dependence on specific initial conditions. Furthermore, while lenient norms demonstrate adaptability in static environments, norms with strict discrimination are shown to be crucial for curbing opportunism and maintaining evolutionary resilience in dynamic settings. Our results reveal the evolutionary dynamics of coupled systems involving reputation institutions and environmental constraints, offering a new theoretical perspective for understanding collective cooperation and social governance in complex environments.

physics.soc-ph↗

arXiv2025

SWE-Compass: Towards Unified Evaluation of Agentic Coding Abilities for Large Language Models

Evaluating large language models (LLMs) for software engineering has been limited by narrow task coverage, language bias, and insufficient alignment with real-world developer workflows. Existing benchmarks often focus on algorithmic problems or Python-centric bug fixing, leaving critical dimensions of software engineering underexplored. To address these gaps, we introduce SWE-Compass1, a comprehensive benchmark that unifies heterogeneous code-related evaluations into a structured and production-aligned framework. SWE-Compass spans 8 task types, 8 programming scenarios, and 10 programming languages, with 2000 high-quality instances curated from authentic GitHub pull requests and refined through systematic filtering and validation. We benchmark ten state-of-the-art LLMs under two agentic frameworks, SWE-Agent and Claude Code, revealing a clear hierarchy of difficulty across task types, languages, and scenarios. Moreover, by aligning evaluation with real-world developer practices, SWE-Compass provides a rigorous and reproducible foundation for diagnosing and advancing agentic coding capabilities in large language models.

cs.SE↗

arXiv2025

KAT-Coder Technical Report

Recent advances in large language models (LLMs) have enabled progress in agentic coding, where models autonomously reason, plan, and act within interactive software development workflows. However, bridging the gap between static text-based training and dynamic real-world agentic execution remains a core challenge. In this technical report, we present KAT-Coder, a large-scale agentic code model trained through a multi-stage curriculum encompassing Mid-Term Training, Supervised Fine-Tuning (SFT), Reinforcement Fine-Tuning (RFT), and Reinforcement-to-Deployment Adaptation. The Mid-Term stage enhances reasoning, planning, and reflection capabilities through a corpus of real software engineering data and synthetic agentic interactions. The SFT stage constructs a million-sample dataset balancing twenty programming languages, ten development contexts, and ten task archetypes. The RFT stage introduces a novel multi-ground-truth reward formulation for stable and sample-efficient policy optimization. Finally, the Reinforcement-to-Deployment phase adapts the model to production-grade IDE environments using Error-Masked SFT and Tree-Structured Trajectory Training. In summary, these stages enable KAT-Coder to achieve robust tool-use reliability, instruction alignment, and long-context reasoning, forming a deployable foundation for real-world intelligent coding agents. Our KAT series 32B model, KAT-Dev, has been open-sourced on https://huggingface.co/Kwaipilot/KAT-Dev.

cs.CL↗

arXiv2025

HiPO: Hybrid Policy Optimization for Dynamic Reasoning in LLMs

Large Language Models (LLMs) increasingly rely on Chain-of-Thought (CoT) reasoning to improve accuracy on complex tasks. However, always generating lengthy reasoning traces is inefficient, leading to excessive token usage and higher inference costs. This paper introduces the Hybrid Policy Optimization (i.e., HiPO), a framework for adaptive reasoning control that enables LLMs to selectively decide when to engage in detailed reasoning (Think-on) and when to respond directly (Think-off). Specifically, HiPO combines a hybrid data pipelineproviding paired Think-on and Think-off responseswith a hybrid reinforcement learning reward system that balances accuracy and efficiency while avoiding over-reliance on detailed reasoning. Experiments across mathematics and coding benchmarks demonstrate that HiPO can substantially reduce token length while maintaining or improving accuracy. Finally, we hope HiPO a can be a principled approach for efficient adaptive reasoning, advancing the deployment of reasoning-oriented LLMs in real-world, resource-sensitive settings.

cs.CL↗

arXiv2025

KAT-V1: Kwai-AutoThink Technical Report

We present Kwaipilot-AutoThink (KAT), an open-source 40B large language model developed to address the overthinking problem in reasoning-intensive tasks, where an automatic thinking training paradigm is proposed to dynamically switch between reasoning and non-reasoning modes based on task complexity. Specifically, first, we construct the dual-regime dataset based on a novel tagging pipeline and a multi-agent synthesis strategy, and then we apply Multi-Token Prediction (MTP)-enhanced knowledge distillation, enabling efficient and fine-grained reasoning transfer with minimal pretraining cost. Besides, we implement a cold-start initialization strategy that introduces mode-selection priors using majority-vote signals and intent-aware prompting. Finally, we propose Step-SRPO, a reinforcement learning algorithm that incorporates intermediate supervision into the GRPO framework, offering structured guidance over both reasoning-mode selection and response accuracy. Extensive experiments across multiple benchmarks demonstrate that KAT consistently matches or even outperforms current state-of-the-art models, including DeepSeek-R1-0528 and Qwen3-235B-A22B, across a wide range of reasoning-intensive tasks while reducing token usage. Notably, KAT outperforms all open-source models and even surpasses o3-mini on the leakage-controlled LiveCodeBench Pro. Beyond academic evaluation, KAT has been successfully deployed in Kwaipilot (i.e., Kuaishou's internal coding assistant), where it improves real-world development workflows with high accuracy, efficiency, and controllable reasoning behaviors. Moreover, we are actively training a 200B Mixture-of-Experts (MoE) model with 40B active parameters, and early results already show significant gains, further demonstrating the scalability of the AutoThink paradigm.

cs.CL↗

arXiv2025

Evolution of cooperation and competition in multilayer networks

Cooperation and competition coexist and coevolve in natural and social systems. Cooperation generates resources, which in turn, drive non-cooperative competition to secure individual shares. How this complex interplay between cooperation and competition shapes the evolution of social dilemmas and welfare remains unknown. In this study, we introduce a two-layer evolutionary game model, in which one layer is a cooperative public goods game, and the other is a competitive involution game, with cross-layer feedback linking the two. We find that feedback can either promote or inhibit cooperation, depending on the baseline conditions. For example, moderate resource and synergy factor values can promote social welfare when feedback strength is large. This provides an approach to adjusting the strength and asymmetry of cross-layer feedback to promote cooperation and social welfare. We thus emphasize the importance of managing feedback mechanisms to balance cooperation and competition in complex social systems.

physics.soc-ph↗

arXiv2024

Evolutionary dynamics in stochastic nonlinear public goods games

Understanding the evolution of cooperation in multiplayer games is of vital significance for natural and social systems. An important challenge is that group interactions often leads to nonlinear synergistic effects. However, previous models mainly focus on deterministic nonlinearity where the arise of synergy or discounting effect is determined by certain conditions, ignoring uncertainty and stochasticity in real-world systems. Here, we develop a probabilistic framework to study the cooperative behavior in stochastic nonlinear public goods games. Through both analytical treatment and Monte Carlo simulations, we provide comprehensive understanding of social dilemmas with stochastic nonlinearity in both well-mixed and structured populations. We find that increasing the degree of nonlinearity makes synergy more advantageous when competing with discounting, thereby promoting cooperation. Interestingly, we show that network reciprocity loses effectiveness when the probability of synergy is small. Moreover, group size exhibits nonlinear effects on group cooperation regardless of the underlying structure. Our findings thus provide novel insights into how stochastic nonlinearity influences the emergence of prosocial behavior.

physics.soc-ph↗

arXiv2023

Reputation-based synergy and discounting mechanism promotes cooperation

A good group reputation often facilitates more efficient synergistic teamwork in production activities. Here we translate this simple motivation into a reputation-based synergy and discounting mechanism in the public goods game. Specifically, the reputation type of a group, either good or bad determined by a reputation threshold, modifies the nonlinear payoff structure described by a unified reputation impact factor. Results show that this reputation-based incentive mechanism could effectively promote cooperation compared with linear payoffs, despite the coexistence of synergy and discounting effects. Notably, the complicated interactions between reputation impact and reputation threshold result in a sharp phase transition from full cooperation to full defection. We also find that the presence of a few discounting groups could increase the average payoffs of cooperators, leading to an interesting phenomenon that when the reputation threshold is raised, the gap between the average payoffs of cooperations and defectors increases while the overall payoff decreases. Our work provides important insights into facilitating cooperation in social groups.

physics.soc-ph↗

arXiv2023

When greediness and self-confidence meet in a social dilemma

A greedy personality is usually accompanied by arrogance and confidence. This work investigates the cooperation success condition in the context of biased payoff allocation and self-confidence. The first component allows the organizer in a spatial public goods game to receive a different proportion of goods than other participants. The second aspect influences the micro-level dynamics of strategy updates, wherein players can maintain their strategy with a certain weight. Analytical results are obtained on square lattices under the weak selection limit. If the organizer attempts to monopolize the public goods, cooperation becomes more attainable. If the confidence increases, cooperation is inhibited. Consequently, these elements have conflicting effects on cooperation, and their simultaneous presence can result in a heterogeneous change of the critical synergy factor. Our theoretical findings underscore the subtle implications of a mutual trait that may manifest as greediness or self-confidence under different circumstances, which are validated through Monte Carlo simulations.

physics.soc-ph↗

arXiv2023

The conflict between self-interaction and updating passivity in the evolution of cooperation

In social dilemmas under weak selection, the capacity for a player to exhibit updating passivity or interact with its own strategy can lead to conflicting outcomes. The central question is which effect is stronger and how their simultaneous presence influences the evolution of cooperation. We introduce a model that considers both effects using different weight factors. We derive theoretical solutions for the conditions of cooperation success and the cooperation level under weak selection, scanning the complete parameter space. When the weight factors are equally strong, the promoting effect of self-interaction to cooperation surpasses the inhibitory effect of updating passivity. Intriguingly, however, we identify non-monotonous cooperation-supporting effects when the weight of updating passivity increases more rapidly. Our findings are corroborated by Monte Carlo simulations and demonstrate robustness across various game types, including the prisoner's dilemma, stag-hunt, and snowdrift games.

cond-mat.stat-mech↗