arXiv subjects

Jeff S. Shamma

Publications and source records attributed to Jeff S. Shamma.

At least 19 recordsLinked to original sources

arXiv2026

Stabilization Limits of Payoff-Based Higher-Order Replicator Dynamics

Replicator dynamics (RD) is a fundamental model in learning in games, connecting evolutionary game theory and online learning. This paper studies payoff-based higher-order variants of RD represented as a cascade interconnection between an integrator in parallel with an auxiliary linear time-invariant (LTI) system and the softmax mapping. We investigate learnability of Nash equilibria under this Nash-stationary learning rule. First, we revisit recent results that establish convergence to Nash Equilibrium whenever the auxiliary LTI system is strictly passive and prove a converse passivity result: if the auxiliary LTI system is not passive, then there exists a static strictly contractive game whose interior Nash equilibrium is unstable under the closed-loop learning dynamics. Second, we show that there exists a class of games with isolated interior Nash equilibria that cannot be locally asymptotically stabilized by any payoff-based higher-order RD whose auxiliary LTI system is asymptotically stable and strictly proper. Finally, we show that if Nash stationarity (i.e., all Nash equilibria are stationary points of the learning dynamics) is relaxed, then generalized exponential RD (Ex-RD) can locally asymptotically stabilize a logit equilibrium for any continuously differentiable game. The stabilized equilibrium can be viewed as an entropy-regularized approximate Nash equilibrium.

eess.SY↗

arXiv2026

Learning Empirical Evidence Equilibria under Weak Environmental Coupling

Strategic multi-agent systems are fundamentally characterized by decentralization, uncertainty, and ambiguity. Agents operating under limited observations will often need to make decisions based on simplified internal models of the environment, reflecting bounded rationality in both computational capacity and environmental knowledge. The Empirical Evidence Equilibrium (EEE) framework explicitly accounts for these limitations by modeling each agent as forming a potentially misspecified belief derived from signals obtained through partial observations of the environment. The resulting equilibrium concept captures the system's steady state under bounded rationality and decentralization. In this work, we study games in which the environment dynamics are driven jointly by exogenous factors and agents' actions. We analyze agent behavior under Q-value iteration where each agent independently forms a belief model, computes Q-values, and derives a greedy strategy, yet the collective actions of all agents jointly shape the environment each agent faces at the next stage. We prove that despite this decentralization, an EEE emerges from the joint dynamics when the coupling between agents' actions and the environment is sufficiently weak. We further extend this result to softmax policies, establishing a contraction result under a sufficient coupling condition.

cs.GT↗

arXiv2026

Higher-Order Uncoupled Learning Dynamics and Nash Equilibrium

We study learnability of mixed-strategy Nash Equilibrium (NE) in general finite games using higher-order replicator dynamics as well as classes of higher-order uncoupled heterogeneous dynamics. In higher-order uncoupled learning dynamics, players have no access to utilities of opponents (uncoupled) but are allowed to use auxiliary states to further process information (higher-order). We establish a link between uncoupled learning and feedback stabilization with decentralized control. Using this association, we show that for any finite game with an isolated completely mixed-strategy NE, there exist higher-order uncoupled learning dynamics that lead (locally) to that NE. We further establish the lack of universality of learning dynamics by linking learning to the control theoretic concept of simultaneous stabilization. We construct two games such that any higher-order dynamics that learn the completely mixed-strategy NE of one of these games can never learn the completely mixed-strategy NE of the other. Next, motivated by imposing natural restrictions on allowable learning dynamics, we introduce the Asymptotic Best Response (ABR) property. Dynamics with the ABR property asymptotically learn a best response in environments that are asymptotically stationary. We show that the ABR property relates to an internal stability condition on higher-order learning dynamics. We provide conditions under which NE are compatible with the ABR property. Finally, we address learnability of mixed-strategy NE in the bandit setting using a bandit version of higher-order replicator dynamics.

cs.MA↗

arXiv2026

Convergence of Payoff-Based Higher-Order Replicator Dynamics in Contractive Games

We study the convergence properties of a payoff-based higher-order version of replicator dynamics, a widely studied model in evolutionary dynamics and game-theoretic learning, in contractive games. Recent work has introduced a control-theoretic perspective for analyzing the convergence of learning dynamics through passivity theory, leading to a classification of learning dynamics based on the passivity notion they satisfy, such as \textdelta-passivity, equilibrium-independent passivity, and incremental passivity. We leverage this framework for the study of higher-order replicator dynamics for contractive games, which form the complement of passive learning dynamics. Standard replicator dynamics can be represented as a cascade interconnection between an integrator and the softmax mapping. Payoff-based higher-order replicator dynamics include a linear time-invariant (LTI) system in parallel with the existing integrator. First, we show that if this added system is strictly passive and asymptotically stable, then the resulting learning dynamics converge locally to the Nash equilibrium in contractive games. Second, we establish global convergence properties using incremental stability analysis for the special case of symmetric matrix contractive games.

eess.SY↗

arXiv2026

Can a Learner Regret Using a No-Regret Algorithm? A Control-Theoretic Study of Performance Dominance

No-regret learning dynamics ensure that a learner asymptotically achieves an average reward no worse than that of any fixed strategy. This no-regret guarantee does not determine the value of the asymptotic average reward. Indeed, it is possible for different no-regret learning dynamics to exhibit different asymptotic average rewards when facing the same environment while both assure the no-regret guarantee. This paper asks whether a "free-lunch" phenomenon can arise among no-regret algorithms. Namely, is it possible for one no-regret learning rule to uniformly outperform another no-regret learning rule across all payoff environments. Stated differently, can a learner regret not using a particular no-regret algorithm? We consider generalized replicator dynamics (RD) as a cascade interconnection between a linear time-invariant (LTI) system and the softmax nonlinearity. Varying this LTI system leads to different realizations of replicator dynamics, including so-called anticipatory RD, exponential RD, and other forms of higher-order RD. Setting the LTI system to be an integrator realizes standard RD, which is known to satisfy the no-regret property. Within this framework, we analyze and compare various realizations of these generalized realizations RD by varying the LTI system. We first formulate performance comparison as a passivity property of an associated comparison system and establish "local" dominance results, i.e., comparing the asymptotic performance near an equilibrium payoff vector. We then cast performance comparison between a form of anticipatory RD and standard RD as an optimal-control problem. We show that the minimal achievable cumulative reward gap is zero, thereby establishing global dominance of anticipatory RD across all payoff environments and establishing a "free lunch" among no-regret learning dynamics.

eess.SY↗

arXiv2025

Passivity, No-Regret, and Convergent Learning in Contractive Games

We investigate the interplay between passivity, no-regret, and convergence in contractive games for various learning dynamic models and their higher-order variants. Our setting is continuous time. Building on prior work for replicator dynamics, we show that if learning dynamics satisfy a passivity condition between the payoff vector and the difference between its evolving strategy and any fixed strategy, then it achieves finite regret. We then establish that the passivity condition holds for various learning dynamics and their higher-order variants. Consequentially, the higher-order variants can achieve convergence to Nash equilibrium in cases where their standard order counterparts cannot, while maintaining a finite regret property. We provide numerical examples to illustrate the lack of finite regret of different evolutionary dynamic models that violate the passivity property. We also examine the fragility of the finite regret property in the case of perturbed learning dynamics. Continuing with passivity, we establish another connection between finite regret and passivity, but with the related equilibrium-independent passivity property. Finally, we present a passivity-based classification of dynamic models according to the various passivity notions they satisfy, namely, incremental passivity, $δ$-passivity, and equilibrium-independent passivity. This passivity-based classification provides a framework to analyze the convergence of learning dynamic models in contractive games.

eess.SY↗

arXiv2024

Distributed Learning Dynamics Converging to the Core of $B$-Matchings

$B$-Matching is a special case of matching problems where nodes can join multiple matchings with the degree of each node constrained by an upper bound, the node's $B$-value. The core solution of a bipartite $B$-matching is both a matching between the nodes respecting the upper bound constraint and an allocation of the weights of the edges among the nodes such that no group of nodes can deviate and collectively gain higher allocation. We present two learning dynamics that converge to the core of the bipartite $B$-matching problems. The first dynamics are centralized dynamics in the nature of the Hungarian method, which converge to the core in a polynomial time. The second dynamics are distributed dynamics, which converge to the core with probability one. For the distributed dynamics, a node maintains only a state consisting of (i) the aspiration levels for all of its possible matches and (ii) the matches, if any, to which it belongs. The node does not keep track of its history nor is it aware of the environment state. In each stage, a randomly activated node proposes to form a new match and changes its aspiration based on the success or failure of its proposal. At this stage, the proposing node inquires about the aspiration of the node it wants to match with to calculate the feasibility of the match. The environment matching structure changes whenever a proposal succeeds. A state is absorbing for the distributed dynamics if and only if it is in the core of the $B$-matching.

cs.GT↗

arXiv2024

Polynomial Lyapunov Functions and Invariant Sets from a New Hierarchy of Quadratic Lyapunov Functions for LTV Systems

We introduce a new class of quadratic functions based on a hierarchy of linear time-varying (LTV) dynamical systems. These quadratic functions in the higher order space can be also seen as a non-homogeneous polynomial Lyapunov functions for the original system, i.e the first system in the hierarchy. These non-homogeneous polynomials are used to obtain accurate outer approximation for the reachable set given the initial condition and less conservative bounds for the impulse response peak of linear, possibly time-varying systems. In addition, we pose an extension to the presented approach to construct invariant sets that are not necessarily Lyapunov functions. The introduced methods are based on elementary linear systems theory and offer very much flexibility in defining arbitrary polynomial Lyapunov functions and invariant sets for LTV systems.

eess.SY↗

arXiv2024

Competition-Based Resilience in Distributed Quadratic Optimization

This paper proposes a novel approach to resilient distributed optimization with quadratic costs in a networked control system (e.g., wireless sensor network, power grid, robotic team) prone to external attacks (e.g., hacking, power outage) that cause agents to misbehave. Departing from classical filtering strategies proposed in literature, we draw inspiration from a game-theoretic formulation of the consensus problem and argue that adding competition to the mix can enhance resilience in the presence of malicious agents. Our intuition is corroborated by analytical and numerical results showing that i) our strategy highlights the presence of a nontrivial tradeoff between blind collaboration and full competition, and ii) such competition-based approach can outperform state-of-the-art algorithms based on Mean Subsequence Reduced.

eess.SY↗

arXiv2023

Distributed Learning Dynamics for Coalitional Games

In the framework of transferable utility coalitional games, a scoring (characteristic) function determines the value of any subset/coalition of agents. Agents decide on both which coalitions to form and the allocations of the values of the formed coalitions among their members. An important concept in coalitional games is that of a core solution, which is a partitioning of agents into coalitions and an associated allocation to each agent under which no group of agents can get a higher allocation by forming an alternative coalition. We present distributed learning dynamics for coalitional games that converge to a core solution whenever one exists. In these dynamics, an agent maintains a state consisting of (i) an aspiration level for its allocation and (ii) the coalition, if any, to which it belongs. In each stage, a randomly activated agent proposes to form a new coalition and changes its aspiration based on the success or failure of its proposal. The coalition membership structure is changed, accordingly, whenever the proposal succeeds. Required communications are that: (i) agents in the proposed new coalition need to reveal their current aspirations to the proposing agent, and (ii) agents are informed if they are joining the proposed coalition or if their existing coalition is broken. The proposing agent computes the feasibility of forming the coalition. We show that the dynamics hit an absorbing state whenever a core solution is reached. We further illustrate the distributed learning dynamics on a multi-agent task allocation setting.

cs.GT↗

arXiv2023

Higher-Order Uncoupled Dynamics Do Not Lead to Nash Equilibrium -- Except When They Do

The framework of multi-agent learning explores the dynamics of how individual agent strategies evolve in response to the evolving strategies of other agents. Of particular interest is whether or not agent strategies converge to well known solution concepts such as Nash Equilibrium (NE). Most "fixed order" learning dynamics restrict an agent's underlying state to be its own strategy. In "higher order" learning, agent dynamics can include auxiliary states that can capture phenomena such as path dependencies. We introduce higher-order gradient play dynamics that resemble projected gradient ascent with auxiliary states. The dynamics are "payoff based" in that each agent's dynamics depend on its own evolving payoff. While these payoffs depend on the strategies of other agents in a game setting, agent dynamics do not depend explicitly on the nature of the game or the strategies of other agents. In this sense, dynamics are "uncoupled" since an agent's dynamics do not depend explicitly on the utility functions of other agents. We first show that for any specific game with an isolated completely mixed-strategy NE, there exist higher-order gradient play dynamics that lead (locally) to that NE, both for the specific game and nearby games with perturbed utility functions. Conversely, we show that for any higher-order gradient play dynamics, there exists a game with a unique isolated completely mixed-strategy NE for which the dynamics do not lead to NE. These results build on prior work that showed that uncoupled fixed-order learning cannot lead to NE in certain instances, whereas higher-order variants can. Finally, we consider the mixed-strategy equilibrium associated with coordination games. While higher-order gradient play can converge to such equilibria, we show such dynamics must be inherently internally unstable.

cs.GT↗

arXiv2023

Can Competition Outperform Collaboration? The Role of Misbehaving Agents

We investigate a novel approach to resilient distributed optimization with quadratic costs in a multi-agent system prone to unexpected events that make some agents misbehave. In contrast to commonly adopted filtering strategies, we draw inspiration from phenomena modeled through the Friedkin-Johnsen dynamics and argue that adding competition to the mix can improve resilience in the presence of misbehaving agents. Our intuition is corroborated by analytical and numerical results showing that (i) there exists a nontrivial trade-off between full collaboration and full competition and (ii) our competition-based approach can outperform state-of-the-art algorithms based on Weighted Mean Subsequence Reduced. We also study impact of communication topology and connectivity on resilience, pointing out insights to robust network design.

eess.SY↗

arXiv2021

RISCuer: A Reliable Multi-UAV Search and Rescue Testbed

We present the Robotics Intelligent Systems & Control (RISC) Lab multiagent testbed for reliable search and rescue and aerial transport in outdoor environments. The system consists of a team of three multirotor unmanned aerial vehicles (UAVs), which are capable of autonomously searching, picking up, and transporting randomly distributed objects in an outdoor field. The method involves vision based object detection and localization, passive aerial grasping with our novel design, GPS based UAV navigation, and safe release of the objects at the drop zone. Our cooperative strategy ensures safe spatial separation between UAVs at all times and we prevent any conflicts at the drop zone using communication enabled consensus. All computation is performed onboard each UAV. We describe the complete software and hardware architecture for the system and demonstrate its reliable performance using comprehensive outdoor experiments, and by comparing our results with some recent, similar works.

cs.RO↗

arXiv2020

Point-to-Point Communication in Integrated Satellite-Aerial Networks: State-of-the-art and Future Challenges

This paper overviews point-to-point (P2P) links for integrated satellite-aerial networks, which are envisioned to be among the key enablers of the sixth-generation (6G) of wireless networks vision. The paper first outlines the unique characteristics of such integrated large-scale complex networks, often denoted by spatial networks, and focuses on two particular space-air infrastructures, namely, satellites networks and high-altitude platforms (HAPs). The paper then classifies the connecting P2P communications links as satellite-to-satellite links at the same layer (SSLL), satellite-to-satellite links at different layers (SSLD), and HAP-to-HAP links (HHL). The paper overviews each layer of such spatial networks separately, and highlights the possible natures of the connecting links (i.e., radio-frequency or free-space optics) with a dedicated overview to the existing link-budget results. The paper, afterwards, presents the prospective merit of realizing such an integrated satellite-HAP network towards providing broadband services in under-served and remote areas. Finally, the paper sheds light on several future research directions in the context of spatial networks, namely large-scale network optimization, intelligent offloading, smart platforms, energy efficiency, multiple access schemes, and distributed spatial networks.

eess.SP↗

arXiv2020

Payoff Dynamics Model and Evolutionary Dynamics Model: Feedback and Convergence to Equilibria

This tutorial article puts forth a framework to analyze the noncooperative strategic interactions among the members of a large population of bounded rationality agents. Our approach hinges on, unifies and generalizes existing methods and models predicated in evolutionary and population games. It does so by adopting a system-theoretic formalism that is well-suited for a broad engineering audience familiar with the basic tenets of nonlinear dynamical systems, Lyapunov stability, storage functions, and passivity. The framework is pertinent for engineering applications in which a large number of agents have the authority to select and repeatedly revise their strategies. A mechanism that is inherent to the problem at hand or is designed and implemented by a coordinator ascribes a payoff to each possible strategy. Typically, the agents will prioritize switching to strategies whose payoff is either higher than the current one or exceeds the population average. The article puts forth a systematic methodology to characterize the stability of the dynamical system that results from the feedback interaction between the payoff mechanism and the revision process. This is important because the set of stable equilibria is an accurate predictor of the population's long-term behavior. The article includes rigorous proofs and examples of application of the stability results, which also extend the state of the art because, unlike previously published work, they allow for a rather general class of dynamical payoff mechanisms. The new results and concepts proposed here are thoroughly compared to previous work, methods and applications of evolutionary and population games.

math.OC↗

arXiv2020

The Optimal and the Greedy: Drone Association and Positioning Schemes for Internet of UAVs

This work considers the deployment of unmanned aerial vehicles (UAVs) over a predefined area to serve a number of ground users. Due to the heterogeneous nature of the network,the UAVs may cause severe interference to the transmissions of each other. Hence, a judicious design of the user-UAV association and UAV locations is desired. A potential game is defined where the players are the UAVs. The potential function is the total sum-rate of the users. The agents utility in the potential games is their marginal contribution to the global welfare or their so-called wonderful life utility. A game-theoretic learning algorithm, binary log-linear learning (BLLL), is then applied to the problem. Given the potential game structure, a consequence of our utility design, the stochastically stable states using BLLL are guaranteed to be the potential maximizers. Hence, we optimally solve the user-UAV association and 3D-location problem. Next, we exploit the sub-modular features of the sum rate function for a given configuration of UAVs to design an efficient greedy algorithm. Despite the simplicity of the greedy algorithm, it comes with a guaranteed performance of $1-1/e$ of the optimal solution. To further reduce the number of iterations, we propose another heuristic greedy algorithm that provides very good results. Our simulations show that, in practice, the proposed greedy approaches achieve significant performance in a few number of iterations.

eess.SP↗

arXiv2019

Network Effects on Robustness of Dynamic Systems

We review selected results related to robustness of networked systems in finite and asymptotically large size regimes, under static and dynamical settings. In the static setting, within the framework of flow over finite networks, we discuss the effect of physical constraints on robustness to loss in link capacities. In the dynamical setting, we review several settings in which small gain type analysis provides tight robustness guarantees for linear dynamics over finite networks towards worst-case and stochastic disturbances. We also discuss network flow dynamic settings where nonlinear techniques facilitate in understanding the effect on robustness of constraints on capacity and information, substituting information with control action, and cascading failure. We also contrast the latter with a representative contagion model. For asymptotically large networks, we discuss the role of network properties in connecting microscopic shocks to emergent macroscopic fluctuations under linear dynamics as well as for economic networks at equilibrium. Through the review of these results, the paper aims to achieve two objectives. First, to highlight selected settings in which the role of interconnectivity structure of a network on its robustness is well-understood. Second, to highlight a few additional settings in which existing system theoretic tools give tight robustness guarantees, and which are also appropriate avenues for future network-theoretic investigations.

eess.SY↗

arXiv2019

usBot: A Modular Robotic Testbed for Programmable Self-Assembly

We present the design, characterization, and experimental results for a new modular robotic system for programmable self-assembly. The proposed system uses the Hybrid Cube Model (HCM), which integrates classical features from both deterministic and stochastic self-organization models. Thus, for instance, the modules are passive as far as their locomotion is concerned (stochastic), and yet they possess an active undocking routine (deterministic). The robots are constructed entirely from readily accessible components and unlike many existing robots, their excitation is not fluid mediated. Instead, the actuation setup is a solid state, independently programmable and highly portable platform. The system is capable of demonstrating fully autonomous and distributed stochastic self-assembly in two dimensions. It is shown to emulate the performance of several existing modular systems and promises to be a substantial effort towards developing a universal testbed for programmable self-assembly.

cs.RO↗