arXiv subjects

Marc Harper

Publications and source records attributed to Marc Harper.

At least 19 recordsLinked to original sources

arXiv2025

Reviving, reproducing, and revisiting Axelrod's second tournament

Direct reciprocity, typically studied using the Iterated Prisoner's Dilemma (IPD), is central to understanding how cooperation evolves. In the 1980s, Robert Axelrod organized two influential IPD computer tournaments, where Tit for Tat (TFT) emerged as the winner. Yet the archival record is incomplete: for the first tournament only a report survives, and for the second the submitted Fortran strategies remain but not the final tournament code. This gap raises questions about the reproducibility of these historically influential results. We recreate the second tournament by restoring the surviving Fortran implementations to compile with modern compilers and by building a Python interface that calls the original strategy functions without modification. Using the open-source Axelrod-Python library to run tournaments, we reproduce Axelrod's main findings: TFT prevails, and successful play tends to be cooperative, responsive to defection, and willing to forgive. Strategy rankings remain mostly unchanged. We then assess the robustness of the originally submitted strategies by incorporating additional strategies, and we run one of the largest IPD tournaments to date. We find that the original tournament was especially favorable to TFT and that it is difficult to dethrone TFT when the original submissions make up the majority of the field. We also observe that several lesser-known submissions perform strongly in more diverse settings and under noise. Our contributions are: (i) the first systematic reproduction of Axelrod's second tournament; (ii) a contemporary reassessment of the original results in light of new strategies and settings; and (iii) a preserved, easy-to-use implementation of the second-tournament strategies within Axelrod-Python to support future research.

cs.GT↗

arXiv2020

Momentum Accelerates Evolutionary Dynamics

We combine momentum from machine learning with evolutionary dynamics, where momentum can be viewed as a simple mechanism of intergenerational memory. Using information divergences as Lyapunov functions, we show that momentum accelerates the convergence of evolutionary dynamics including the replicator equation and Euclidean gradient descent on populations. When evolutionarily stable states are present, these methods prove convergence for small learning rates or small momentum, and yield an analytic determination of the relative decrease in time to converge that agrees well with computations. The main results apply even when the evolutionary dynamic is not a gradient flow. We also show that momentum can alter the convergence properties of these dynamics, for example by breaking the cycling associated to the rock-paper-scissors landscape, leading to either convergence to the ordinarily non-absorbing equilibrium, or divergence, depending on the value and mechanism of momentum.

cs.LG↗

arXiv2020

Properties of Winning Iterated Prisoner's Dilemma Strategies

Researchers have explored the performance of Iterated Prisoner's Dilemma strategies for decades, from the celebrated performance of Tit for Tat to the introduction of the zero-determinant strategies and the use of sophisticated learning structures such as neural networks. Many new strategies have been introduced and tested in a variety of tournaments and population dynamics. Typical results in the literature, however, rely on performance against a small number of somewhat arbitrarily selected strategies in a small number of tournaments, casting doubt on the generalizability of conclusions. In this work, we analyze a large collection of 195 strategies in thousands of computer tournaments, present the top performing strategies across multiple tournament types, and distill their salient features. The results show that there is not yet a single strategy that performs well in diverse Iterated Prisoner's Dilemma scenarios, nevertheless there are several properties that heavily influence the best performing strategies. This refines the properties described by Axelrod in light of recent and more diverse opponent populations to: be nice, be provocable and generous, be a little envious, be clever, and adapt to the environment. More precisely, we find that strategies perform best when their probability of cooperation matches the total tournament population's aggregate cooperation probabilities. The features of high performing strategies help cast some light on why strategies such as Tit For Tat performed historically well in tournaments and why zero-determinant strategies typically do not fare well in tournament settings. Furthermore, our findings have implications for the future training of autonomous agents, as understanding the crucial features for incorporation into these agents becomes essential.

cs.GT↗

arXiv2019

Memory depth of finite state machine strategies for the iterated prisoner's dilemma

We develop an efficient algorithm to determine the memory-depth of finite state machines and apply the algorithm to a collection of iterated prisoner's dilemma strategies. The calculation agrees with the memory-depth of other representations of common strategies such as Tit-For-Tat, Tit-For-2-Tats, etc. which are typically represented by lookup tables. Our algorithm allows the complexity of finite state machine based strategies to be characterized on the same footing as memory-n strategies.

cs.GT↗

arXiv2019

User Sentiment as a Success Metric: Persistent Biases Under Full Randomization

We study user sentiment (reported via optional surveys) as a metric for fully randomized A/B tests. Both user-level covariates and treatment assignment can impact response propensity. We propose a set of consistent estimators for the average and local treatment effects on treated and respondent users. We show that our problem can be mapped onto the intersection of the missing data problem and observational causal inference, and we identify conditions under which consistent estimators exist. We evaluate the performance of estimators via simulation studies and find that more complicated models do not necessarily provide superior performance.

stat.ME↗

arXiv2019

Recognising and evaluating the effectiveness of extortion in the Iterated Prisoner's Dilemma

Since the introduction of zero-determinant strategies, extortionate strategies have received considerable interest. While an interesting class of strategies, the definitions of extortionate strategies are algebraically rigid, apply only to memory-one strategies, and require complete knowledge of a strategy (memory-one cooperation probabilities). We describe a method to detect extortionate behaviour from the history of play of a strategy. When applied to a corpus of 204 strategies this method detects extortionate behaviour in well-known extortionate strategies as well others that do not fit the algebraic definition. The highest performing strategies in this corpus are able to exhibit selectively extortionate behavior, cooperating with strong strategies while exploiting weaker strategies, which no memory-one strategy can do. These strategies emerged from an evolutionary selection process and their existence contradicts widely-repeated folklore in the evolutionary game theory literature: complex strategies can be extraordinarily effective, zero-determinant strategies can be outperformed by non-zero determinant strategies, and longer memory strategies are able to outperform short memory strategies. Moreover, while resistance to extortion is critical for the evolution of cooperation, the extortion of weak opponents need not prevent cooperation between stronger opponents, and this adaptability may be crucial to maintaining cooperation in the long run.

cs.GT↗

arXiv2017

Evolution Reinforces Cooperation with the Emergence of Self-Recognition Mechanisms: an empirical study of the Moran process for the iterated Prisoner's dilemma

We present insights and empirical results from an extensive numerical study of the evolutionary dynamics of the iterated prisoner's dilemma. Fixation probabilities for Moran processes are obtained for all pairs of 164 different strategies including classics such as TitForTat, zero determinant strategies, and many more sophisticated strategies. Players with long memories and sophisticated behaviours outperform many strategies that perform well in a two player setting. Moreover we introduce several strategies trained with evolutionary algorithms to excel at the Moran process. These strategies are excellent invaders and resistors of invasion and in some cases naturally evolve handshaking mechanisms to resist invasion. The best invaders were those trained to maximize total payoff while the best resistors invoke handshake mechanisms. This suggests that while maximizing individual payoff can lead to the evolution of cooperation through invasion, the relatively weak invasion resistance of payoff maximizing strategies are not as evolutionarily stable as strategies employing handshake mechanisms.

cs.GT↗

arXiv2017

Reinforcement Learning Produces Dominant Strategies for the Iterated Prisoner's Dilemma

We present tournament results and several powerful strategies for the Iterated Prisoner's Dilemma created using reinforcement learning techniques (evolutionary and particle swarm algorithms). These strategies are trained to perform well against a corpus of over 170 distinct opponents, including many well-known and classic strategies. All the trained strategies win standard tournaments against the total collection of other opponents. The trained strategies and one particular human made designed strategy are the top performers in noisy tournaments also.

cs.GT↗

arXiv2016

An open reproducible framework for the study of the iterated prisoner's dilemma

The Axelrod library is an open source Python package that allows for reproducible game theoretic research into the Iterated Prisoner's Dilemma. This area of research began in the 1980s but suffers from a lack of documentation and test code. The goal of the library is to provide such a resource, with facilities for the design of new strategies and interactions between them, as well as conducting tournaments and ecological simulations for populations of strategies. With a growing collection of 139 strategies, the library is a also a platform for an original tournament that, in itself, is of interest to the game theoretic community. This paper describes the Iterated Prisoner's Dilemma, the Axelrod library and its development, and insights gained from some novel research.

cs.GT↗

arXiv2015

Entropic Equilibria Selection of Stationary Extrema in Finite Populations

We propose the entropy of random Markov trajectories originating and terminating at a state as a measure of the stability of a state of a Markov process. These entropies can be computed in terms of the entropy rates and stationary distributions of Markov processes. We apply this definition of stability to local maxima and minima of the stationary distribution of the Moran process with mutation and show that variations in population size, mutation rate, and strength of selection all affect the stability of the stationary extrema.

math.DS↗

arXiv2014

The Art of War: Beyond Memory-one Strategies in Population Games

We define a new strategy for population games based on techniques from machine learning and statistical inference that is essentially uninvadable and can successfully invade (significantly more likely than a neutral mutant) essentially all known memory-one strategies for the prisoner's dilemma and other population games, including ALLC (always cooperate), ALLD (always defect), tit-for-tat (TFT), win-stay-lose-shift (WSLS), and zero determinant (ZD) strategies, including extortionate and generous strategies. We will refer to a player using this strategy as an "information player" and the specific implementation as $IP_0$. Such players use the history of play to identify opponent's strategies and respond accordingly, and naturally learn to cooperate with each other.

q-bio.PE↗

arXiv2014

Entropy Rates of the Multidimensional Moran Processes and Generalizations

The interrelationships of the fundamental biological processes natural selection, mutation, and stochastic drift are quantified by the entropy rate of Moran processes with mutation, measuring the long-run variation of a Markov process. The entropy rate is shown to behave intuitively with respect to evolutionary parameters such as monotonicity with respect to mutation probability (for the neutral landscape), relative fitness, and strength of selection. Strict upper bounds, depending only on the number of replicating types, for the entropy rate are given and the neutral fitness landscape attains the maximum in the large population limit. Various additional limits are computed including small mutation, weak and strong selection, and large population holding the other parameters constant, revealing the individual contributions and dependences of each evolutionary parameter on the long-run outcomes of the processes.

math.DS↗

arXiv2013

Detecting Concepts Crucial for Success in Mathematics Courses from Knowledge State-based Placement Data

We show that individual topics and skills can have a dramatic effect on the outcomes of students in various mathematics courses at the University of Illinois. Data from the placement program at Illinois associates a knowledge state, a subset of 182 items and skills that a student is able to complete successfully and repeatedly, with their final grades in a variety of courses from college algebra through multivariate calculus. Using various conditional probabilities and odds ratios, we classify items based on their association with successful and unsuccessful course outcomes, showing that some skills that are advanced for some courses are fundamental or basic to more advanced courses. We examine the impact of specific items across the courses in the traditional college algebra, precalculus, and calculus sequence, as well as courses not typically covered by placement programs, such as higher calculus courses. Visualizations of the knowledge states associated to each student are given for some specific courses and for the entire collection of placement courses, allowing the impact of specific topics to be examined across the undergraduate curriculum.

math.HO↗

arXiv2013

Stationary Stability for Evolutionary Dynamics in Finite Populations

We demonstrate a vast expansion of the theory of evolutionary stability to finite populations with mutation, connecting the theory of the stationary distribution of the Moran process with the Lyapunov theory of evolutionary stability. We define the notion of stationary stability for the Moran process with mutation and generalizations, as well as a generalized notion of evolutionary stability that includes mutation called an incentive stable state (ISS) candidate. For sufficiently large populations, extrema of the stationary distribution are ISS candidates and we give a family of Lyapunov quantities that are locally minimized at the stationary extrema and at ISS candidates. In various examples, including for the Moran and Wright-Fisher processes, we show that the local maxima of the stationary distribution capture the traditionally-defined evolutionarily stable states. The classical stability theory of the replicator dynamic is recovered in the large population limit. Finally we include descriptions of possible extensions to populations of variable size and populations evolving on graphs.

math.DS↗

arXiv2013

Incentive Processes in Finite Populations

We define the incentive process, a natural generalization of the Moran process incorporating evolutionary updating mechanisms corresponding to well-known evolutionary dynamics, such as the logit, projection, and best-reply dynamics. Fixation probabilities and internal stable states are given for a variety of incentives, including new closed-forms, as well as results relating fixation probabilities for members of two one-parameter families of incentive processes. We show that the behaviors of the incentive process can deviate significantly from the analogous properties of deterministic evolutionary dynamics in some ways but are similar in others. For example, while the fixation probabilities change, their ratio remains constant.

math.DS↗

arXiv2013

Mean Evolutionary Dynamics for Stochastically Switching Environments

Populations of replicating entities frequently experience sudden or cyclical changes in environment. We explore the implications of this phenomenon via a environmental switching parameter in several common evolutionary dynamics models including the replicator dynamic for linear symmetric and asymmetric landscapes, the Moran process, and incentive dynamics. We give a simple relationship between the probability of environmental switching, the relative fitness gain, and the effect on long term behavior in terms of fixation probabilities and long term outcomes for deterministic dynamics. We also discuss cases where the dynamic changes, for instance a population evolving under a replicator dynamic switching to a best-reply dynamic and vice-versa, giving Lyapunov stability results.

math.DS↗

arXiv2013

Inferring Fitness in Finite Populations with Moran-like dynamics

Biological fitness is not an observable quantity and must be inferred from population dynamics. Bayesian inference applied to the Moran process and variants yields a robust inference method that can infer fitness in populations evolving via a Moran dynamic and generalizations. Information about fitness is derived solely from birth-events in birth-death and death-birth processes in which selection acts proportionally to fitness, which allows the method to be applied to populations on a network where the network itself may be changing in time. Populations may also be allowed to change size while still allowing estimates for fitness to be inferred.

math.DS↗

arXiv2013

The Inherent Randomness of Evolving Populations

The entropy rates of the Wright-Fisher process, the Moran process, and generalizations are computed and used to compare these processes and their dependence on standard evolutionary parameters. Entropy rates are measures of the variation dependent on both short-run and long-run behavior, and allow the relationships between mutation, selection, and population size to be examined. Bounds for the entropy rate are given for the Moran process (independent of population size) and for the Wright-Fisher process (bounded for fixed population size). A generational Moran process is also presented for comparison to the Wright-Fisher Process. Results include analytic results and computational extensions.

math.DS↗