arXiv subjects

Guozhong Zheng

Publications and source records attributed to Guozhong Zheng.

18 recordsLinked to original sources

arXiv2026

A brief review of evolutionary game dynamics in the reinforcement learning paradigm

Cooperation, fairness, trust, and resource coordination are cornerstones of modern civilization, yet their emergence remains inadequately explained by the persistent discrepancies between theoretical predictions and behavioral experiments. Part of this gap may arise from the imitation learning paradigm commonly used in prior theoretical models, which assumes individuals merely copy successful neighbors according to predetermined, fixed rules. This review examines recent advances in evolutionary game dynamics that employ reinforcement learning (RL) as an alternative paradigm. In RL, individuals learn through trial and error and introspectively refine their strategies based on environmental feedback. We begin by introducing key concepts in evolutionary game theory and the two learning paradigms, then synthesize progress in applying RL to elucidate cooperation, trust, fairness, optimal resource coordination, and ecological dynamics. Collectively, these studies indicate that RL offers a promising unified framework for understanding the diverse social and ecological phenomena observed in human and natural systems.

q-bio.PE↗

arXiv2025

Optimal coordination of resources: A solution from reinforcement learning

Efficient allocation is important in nature and human society, where individuals frequently compete for limited resources. The Minority Game (MG) is perhaps the simplest toy model to address this issue. However, most previous solutions assume that the strategies are provided a priori and static, failing to capture their adaptive nature. Here, we introduce the reinforcement learning (RL) paradigm to MG, where individuals adjust decisions based on accumulated experience and expected rewards dynamically. We find that this RL framework achieves optimal resource coordination when individuals balance the exploitation of experience with random exploration. Yet, the imbalanced strategies of the two lead to suboptimal partial coordination or even anti-coordination. Our mechanistic analysis reveals a symmetry-breaking in action preferences at the optimum, offering a fresh solution to the MG and new insights into the resource allocation problem.

physics.soc-ph↗

arXiv2025

Evolution of cooperation with Q-learning: the impact of information perception

The inherent complexity of human beings manifests in a remarkable diversity of responses to intricate environments, enabling us to approach problems from varied perspectives. However, in the study of cooperation, existing research within the reinforcement learning framework often assumes that individuals have access to identical information when making decisions, which contrasts with the reality that individuals frequently perceive information differently. In this study, we employ the Q-learning algorithm to explore the impact of information perception on the evolution of cooperation in a two-person Prisoner's Dilemma game. We demonstrate that the evolutionary processes differ significantly across three distinct information perception scenarios, highlighting the critical role of information structure in the emergence of cooperation. Notably, the asymmetric information scenario reveals a complex dynamical process, including the emergence, breakdown, and reconstruction of cooperation, mirroring psychological shifts observed in human behavior. Our findings underscore the importance of information structure in fostering cooperation, offering new insights into the establishment of stable cooperative relationships among humans.

q-bio.PE↗

arXiv2025

Evolution of cooperation in a bimodal mixture of conditional cooperators

Extensive behavioral experiments reveal that conditional cooperation is a prevalent phenomenon. Previous game-theoretical studies have predominantly relied on hard-manner models, where cooperation is triggered only upon reaching a specific threshold. However, this approach contrasts with the observed flexibility of human behaviors, where individuals adapt their strategies dynamically based on their surroundings. To capture this adaptability, we introduce a soft form of conditional cooperation by integrating the Q-learning algorithm from reinforcement learning. In this form, players not only reciprocate mutual cooperation but may also defect in highly cooperative environments or cooperate in less cooperative settings to maximize rewards. To explore the effects of hard and soft conditional cooperators, we examine their interactions in two scenarios: structural mixture (SM) and probabilistic mixture (PM), where the two behavioral modes are fixed and probabilistically adopted, respectively. In SM, hard conditional cooperators enhance cooperation when the threshold is low but hinder it otherwise. Surprisingly, in PM, the cooperation prevalence exhibits two first-order phase transitions as the probability is varied, leading to high, low, and vanishing levels of cooperation. Analysis of Q-tables offers insights into the "psychological shifts" of soft conditional cooperators and the overall evolutionary dynamics. Model extensions confirm the robustness of our findings. These results highlight the novel complexities arising from the diversity of conditional cooperators.

q-bio.PE↗

arXiv2024

Decoding fairness: a reinforcement learning perspective

Behavioral experiments on the ultimatum game (UG) reveal that we humans prefer fair acts, which contradicts the prediction made in orthodox Economics. Existing explanations, however, are mostly attributed to exogenous factors within the imitation learning framework. Here, we adopt the reinforcement learning paradigm, where individuals make their moves aiming to maximize their accumulated rewards. Specifically, we apply Q-learning to UG, where each player is assigned two Q-tables to guide decisions for the roles of proposer and responder. In a two-player scenario, fairness emerges prominently when both experiences and future rewards are appreciated. In particular, the probability of successful deals increases with higher offers, which aligns with observations in behavioral experiments. Our mechanism analysis reveals that the system undergoes two phases, eventually stabilizing into fair or rational strategies. These results are robust when the rotating role assignment is replaced by a random or fixed manner, or the scenario is extended to a latticed population. Our findings thus conclude that the endogenous factor is sufficient to explain the emergence of fairness, exogenous factors are not needed.

cs.LG↗

arXiv2024

Multiple sliding ferroelectricity of rhombohedral-stacked InSe for reconfigurable photovoltaics and imaging applications

Through stacking engineering of two-dimensional (2D) materials, a switchable interface polarization can be generated through interlayer sliding, so called sliding ferroelectricity, which is advantageous over the traditional ferroelectricity due to ultra-thin thickness, high switching speed and low fatigue. However, 2D materials with intrinsic sliding ferroelectricity are still rare, with the exception of rhombohedral-stacked MoS2, which limits sliding ferroelectricity for practical applications such as high-speed storage, photovoltaic, and neuromorphic computing. Here, we reported the observation of sliding ferroelectricity with multiple states in undoped rhombohedral-stacked InSe (γ-InSe) via dual-frequency resonance tracking piezoresponse force microscopy, scanning Kelvin probe microscopy and conductive atomic force microscopy. The tunable bulk photovoltaic effect via the electric field is achieved in the graphene/γ-InSe/graphene tunneling device with a photovoltaic current density of ~15 mA/cm2, which is attributed to the multiple sliding steps in γ-InSe according to our theoretical calculations. The vdw tunneling device also features a high photo responsivity of ~255 A/W and a fast response time for real-time imaging. Our work not only enriches rhombohedral-stacked 2D materials for sliding ferroelectricity, but also sheds light on their potential for tunable photovoltaics and imaging applications.

cond-mat.mtrl-sci↗

arXiv2024

Evolution of cooperation in the public goods game with Q-learning

Recent paradigm shifts from imitation learning to reinforcement learning (RL) is shown to be productive in understanding human behaviors. In the RL paradigm, individuals search for optimal strategies through interaction with the environment to make decisions. This implies that gathering, processing, and utilizing information from their surroundings are crucial. However, existing studies typically study pairwise games such as the prisoners' dilemma and employ a self-regarding setup, where individuals play against one opponent based solely on their own strategies, neglecting the environmental information. In this work, we investigate the evolution of cooperation with the multiplayer game -- the public goods game using the Q-learning algorithm by leveraging the environmental information. Specifically, the decision-making of players is based upon the cooperation information in their neighborhood. Our results show that cooperation is more likely to emerge compared to the case of imitation learning by using Fermi rule. Of particular interest is the observation of an anomalous non-monotonic dependence which is revealed when voluntary participation is further introduced. The analysis of the Q-table explains the mechanisms behind the cooperation evolution. Our findings indicate the fundamental role of environment information in the RL paradigm to understand the evolution of cooperation, and human behaviors in general.

q-bio.PE↗

arXiv2024

Catalytic evolution of cooperation in a population with behavioural bimodality

The remarkable adaptability of humans in response to complex environments is often demonstrated by the context-dependent adoption of different behavioral modes. However, the existing game-theoretic studies mostly focus on the single-mode assumption, and the impact of this behavioral multimodality on the evolution of cooperation remains largely unknown. Here, we study how cooperation evolves in a population with two behavioral modes. Specifically, we incorporate Q-learning and Tit-for-Tat (TFT) rules into our toy model, where prisoner's dilemma game is played and we investigate the impact of the mode mixture on the evolution of cooperation. While players in Q-learning mode aim to maximize their accumulated payoffs, players within TFT mode repeat what their neighbors have done to them. In a structured mixing implementation where the updating rule is fixed for each individual, we find that the mode mixture greatly promotes the overall cooperation prevalence. The promotion is even more significant in the probabilistic mixing, where players randomly select one of the two rules at each step. Finally, this promotion is robust when players are allowed to adaptively choose the two modes by real-time comparison. In all three scenarios, players within the Q-learning mode act as catalyzer that turns the TFT players to be more cooperative, and as a result drive the whole population to be highly cooperative. The analysis of Q-tables explains the underlying mechanism of cooperation promotion, which captures the ``psychologic evolution" in the players' mind. Our study indicates that the variety of behavioral modes is non-negligible, and could be crucial to clarify the emergence of cooperation in the real world.

q-bio.PE↗

arXiv2024

Emergence of cooperation under punishment: A reinforcement learning perspective

Punishment is a common tactic to sustain cooperation and has been extensively studied for a long time. While most of previous game-theoretic work adopt the imitation learning where players imitate the strategies who are better off, the learning logic in the real world is often much more complex. In this work, we turn to the reinforcement learning paradigm, where individuals make their decisions based upon their past experience and long-term returns. Specifically, we investigate the Prisoners' dilemma game with Q-learning algorithm, and cooperators probabilistically pose punishment on defectors in their neighborhood. Interestingly, we find that punishment could lead to either continuous or discontinuous cooperation phase transitions, and the nucleation process of cooperation clusters is reminiscent of the liquid-gas transition. The uncovered first-order phase transition indicates that great care needs to be taken when implementing the punishment compared to the continuous scenario.

q-bio.PE↗

arXiv2023

Decoding trust: A reinforcement learning perspective

Behavioral experiments on the trust game have shown that trust and trustworthiness are universal among human beings, contradicting the prediction by assuming \emph{Homo economicus} in orthodox Economics. This means some mechanism must be at work that favors their emergence. Most previous explanations however need to resort to some factors based upon imitative learning, a simple version of social learning. Here, we turn to the paradigm of reinforcement learning, where individuals update their strategies by evaluating the long-term return through accumulated experience. Specifically, we investigate the trust game with the Q-learning algorithm, where each participant is associated with two evolving Q-tables that guide one's decision making as trustor and trustee respectively. In the pairwise scenario, we reveal that high levels of trust and trustworthiness emerge when individuals appreciate both their historical experience and returns in the future. Mechanistically, the evolution of the Q-tables shows a crossover that resembles human's psychological changes. We also provide the phase diagram for the game parameters, where the boundary analysis is conducted. These findings are robust when the scenario is extended to a latticed population. Our results thus provide a natural explanation for the emergence of trust and trustworthiness without external factors involved. More importantly, the proposed paradigm shows the potential in deciphering many puzzles in human behaviors.

q-bio.PE↗

arXiv2023

A Generative Hypergraph Model for Double Heterogeneity

While network science has become an indispensable tool for studying complex systems, the conventional use of pairwise links often shows limitations in describing high-order interactions properly. Hypergraphs, where each edge can connect more than two nodes, have thus become a new paradigm in network science. Yet, we are still in lack of models linking network growth and hyperedge expansion, both of which are commonly observable in the real world. Here, we propose a generative hypergraph model by employing the preferential attachment mechanism in both nodes and hyperedge formation. The model can produce bi-heterogeneity, exhibiting scale-free distributions in both hyperdegree and hyperedge size. We provide a mean-field treatment that gives the expression of the two scaling exponents, which agree with the numerical simulations. Our model may help to understand the networked systems showing both types of heterogeneity and facilitate the study of complex dynamics thereon.

physics.soc-ph↗

arXiv2023

Emergence of cooperation in a population with bimodal response behaviors

We human beings show remarkable adaptability in response to complex surroundings, we adopt different behavioral modes at different occasions, such response multimodality is critical to our survival. Yet, how this behavioral multimodality affects the evolution of cooperation remains largely unknown. Here we build a toy model to address this issue by considering a population with bimodal response behaviors, or specifically, with the Fermi and Tit-for-tat updating rules. While the former rule tends to imitate the strategies of those neighbors who are doing well, the latter repeats what their neighbors did to them. In a structural mixing implementation, where the updating rule is fixed for each individual, we find that a moderate mode mixture unexpectedly boosts the overall cooperation level of the population. The boost is even more pronounced in the probabilistic mixing, where each individual randomly chooses one of the two modes at each step, and full cooperation is seen in a wide range. These findings are robust to the underlying topology of the population. Our mean-field treatment reveals that the cooperation prevalence within the players with the Fermi rule linearly increases with the fraction of TFT players and explains the non-monotonic dependence in the structural mixing. Our study shows that the diversity in response behaviors may help to explain the emergence of cooperation in realistic contexts.

physics.soc-ph↗

arXiv2023

Pinning control of social fairness in the Ultimatum game

Decent social fairness is highly desired both for socio-economic activities and individuals, as it is one of the cornerstones of our social welfare and sustainability. How to effectively promote the level of fairness thus becomes a significant issue to be addressed. Here, by adopting a pinning control procedure, we find that when a very small fraction of individuals are pinned to be fair players in the Ultimatum Game, the whole population unexpectedly evolves into the full fairness level. The basic observations are quite robust in homogeneous networks, but the converging time as a function of the pinning number shows different laws for different underlying topologies. For heterogeneous networks, this leverage effect is even more pronounced that one hub node is sufficient for the aim, and a periodic on-off control procedure can be applied to further save the control cost. Intermittent failures are seen when the pinning control is marginally strong, our statistical analysis indicates some sort of criticality. Our work suggests that the pinning control procedure could potentially be a good strategy to promote the social fairness for some real scenarios when necessary.

physics.soc-ph↗

arXiv2022

Oscillatory cooperation prevalence emerges from misperception

Oscillatory behaviors are ubiquitous in nature and the human society. However, most previous works fail to reproduce them in the two-strategy game-theoretical models. Here we show that oscillatory behaviors naturally emerge if incomplete information is incorporated into the cooperation evolution of a non-Markov model. Specifically, we consider a population playing prisoner's dilemma game, where each individual can only probabilistically get access to their neighbors' payoff information and store them within their memory with a given length. They make their decisions based upon these memories. Interestingly, we find that the level of cooperation generally cannot stabilize but render quasi-periodic oscillation, and this observation is strengthened for a longer memory and a smaller information acquisition probability. The mechanism uncovered shows that there are misperceived payoffs about the player's neighborhood, facilitating the growth of cooperators and defectors at different stages that leads to oscillatory behaviors as a result. Our findings are robust to the underlying structure of the population. Given the omnipresence of incomplete information, our findings may provide a plausible explanation for the phenomenon of oscillatory behaviors in the real world.

q-bio.PE↗

arXiv2022

Probabilistic fair behaviors spark its boost in the Ultimatum Game: the strength of good Samaritans

Behavioral experiments on the Ultimatum Game have shown that we human beings have remarkable preference in fair play, contradicting the predictions by the game theory. Most of the existing models seeking for explanations, however, strictly follow the assumption of \emph{Homo economicus} in orthodox Economics that people are self-interested and fully rational to maximize their earnings. Here we relax this assumption by allowing that people probabilistically choose to be "good Samaritans", acting as fair players from time to time. For well-mixed and homogeneously structured populations, we numerically show that as this probability increases the level of fairness undergoes from the low scenario abruptly to the full fairness state, where occasional fair behaviors ($\sim5\%$) are sufficient to drive the whole population to behave in the half-half split manner. We also develop a mean-field theory, which correctly reproduces the first-order phase transition and points out that the bistability is an intrinsic property of this game and small fair acts lead to dramatical change due to its bifurcation structure. Heterogeneously structured populations, however, display continuous fairness transition; surprisingly, very few hub nodes acting as fair players are able to entrain the whole population to the full fairness state. Our results thus reveal the unexpected strength of "good Samaritans", which may constitute a new explanation for the emergence of fairness in our society.

q-bio.PE↗

arXiv2021

Emergent route towards cooperation in interacting games: the dynamical reciprocity

The success of modern civilization is built upon widespread cooperation in human society, deciphering the mechanisms behind has being a major goal for centuries. A crucial fact is, however, largely missing in most prior studies that games in the real world are typically played simultaneously and interactively rather than separately as assumed. Here we introduce the idea of interacting games that different games coevolve and influence each other's decision-making. We show that as the game-game interaction becomes important, the cooperation phase transition dramatically improves, a fairly high level of cooperation is reached for all involved games when interaction goes to be strong. A mean-field theory indicates that a new mechanism -- \emph{the dynamical reciprocity}, as a counterpart to the well-known network reciprocity, is at work to foster cooperation, which is confirmed by the detailed analysis. This revealed reciprocity is robust against variations in the game type, the population structure, and the updating rules etc, and more games generally yield a higher level of cooperation. Our findings point out the great potential towards high cooperation for many issues are interwoven with each other in the real world, and also the possibility of sustaining decent cooperation even in extremely adverse circumstances.

physics.soc-ph↗

arXiv2021

Dynamical reciprocity in interacting games: numerical results and mechanism analysis

We study the evolution of two mutually interacting games with both pairwise games as well as the public goods game on different topologies. On 2d square lattices, we reveal that the game-game interaction can promote the cooperation prevalence in all cases, and the cooperation-defection phase transitions even become absent and fairly high cooperation is expected when the interaction goes to be very strong. A mean-field theory is developed that points out new dynamical routes arising therein. Detailed analysis shows indeed that there are rich categories of interactions in either individual or bulk scenario: invasion, neutral, and catalyzed types; their combination puts cooperators at a persistent advantage position, which boosts the cooperation. The robustness of the revealed reciprocity is strengthened by the studies of model variants, including asymmetrical or time-varying interactions, games of different types, games with time-scale separation, different updating rules etc. The structural complexities of the underlying population, such as Newman--Watts small world networks, Erdős--Rényi random networks, and Barabási--Albert networks, also do not alter the working of the dynamical reciprocity. In particular, as the number of games engaged increases, the cooperation level continuously improves in general. Our work thus uncovers a new class of cooperation mechanism and indicates the great potential for human cooperation where concurrent issues are so often seen in the real world.

physics.soc-ph↗

arXiv2021

Social hierarchy promotes the cooperation prevalence

Social hierarchy is important that can not be ignored in human socioeconomic activities and in the animal world. Here we incorporate this factor into the evolutionary game to see what impact it could have on the cooperation outcome. The probabilistic strategy adoption between two players is then not only determined by their payoffs, but also by their hierarchy difference -- players in the high rank are more likely to reproduce their strategies than the peers in the low rank. Through simulating the evolution of Prisoners' dilemma game with three hierarchical distributions, we find that the levels of cooperation are enhanced in all cases, and the enhancement is optimal in the uniform case. The enhancement is due to the fact that the presence of hierarchy facilitates the formation of cooperation clusters with high-rank players acting as the nucleation cores. This mechanism remains valid on Barabási-Albert scale-free networks, in particular the cooperation enhancement is maximal when the hubs are of higher social ranks. We also study a two-hierarchy model, where similar cooperation promotion is revealed and some theoretical analyses are provided. Our finding may partially explain why the social hierarchy is so ubiquitous on this planet.

physics.soc-ph↗