arXiv subjects

Ceyhun Eksin

Publications and source records attributed to Ceyhun Eksin.

At least 19 recordsLinked to original sources

arXiv2026

Almost Sure Convergence of Networked Policy Gradient over Time-Varying Networks in Markov Potential Games

We propose networked policy gradient play for solving Markov potential games with continuous and/or discrete state-action pairs. During the game, agents use parametrized and differentiable policies that depend on the current state and the policy parameters of other agents. During training, agents update their policy parameters following stochastic gradients. The gradient estimation involves two consecutive episodes, generating unbiased estimators of reward and policy score functions. In addition, it involves keeping estimates of others' parameters using consensus steps given local estimates received through a time-varying communication network. In Markov potential games, there exists a potential value function among agents with gradients corresponding to the gradients of local value functions. Using this structure, we prove almost sure convergence to a stationary point of the potential value function with rate $O(1/ε^2)$. Compared to previous works, our results do not require bounded policy gradients or initial agreement on the values of individual policy parameters. Numerical experiments on a dynamic multi-agent newsvendor problem verify the convergence of local beliefs and gradients. It further shows that networked policy gradient play converges as fast as independent policy gradient updates, while collecting higher rewards.

eess.SY↗

arXiv2025

The Lagrangian Method for Solving Constrained Markov Games

We propose the concept of a Lagrangian game to solve constrained Markov games. Such games model scenarios where agents face cost constraints in addition to their individual rewards, that depend on both agent joint actions and the evolving environment state over time. Constrained Markov games form the formal mechanism behind safe multiagent reinforcement learning, providing a structured model for dynamic multiagent interactions in a multitude of settings, such as autonomous teams operating under local energy and time constraints, for example. We develop a primal-dual approach in which agents solve a Lagrangian game associated with the current Lagrange multiplier, simulate cost and reward trajectories over a fixed horizon, and update the multiplier using accrued experience. This update rule generates a new Lagrangian game, initiating the next iteration. Our key result consists in showing that the sequence of solutions to these Lagrangian games yields a nonstationary Nash solution for the original constrained Markov game.

math.OC↗

arXiv2024

Simulation-Based Optimistic Policy Iteration For Multi-Agent MDPs with Kullback-Leibler Control Cost

This paper proposes an agent-based optimistic policy iteration (OPI) scheme for learning stationary optimal stochastic policies in multi-agent Markov Decision Processes (MDPs), in which agents incur a Kullback-Leibler (KL) divergence cost for their control efforts and an additional cost for the joint state. The proposed scheme consists of a greedy policy improvement step followed by an m-step temporal difference (TD) policy evaluation step. We use the separable structure of the instantaneous cost to show that the policy improvement step follows a Boltzmann distribution that depends on the current value function estimate and the uncontrolled transition probabilities. This allows agents to compute the improved joint policy independently. We show that both the synchronous (entire state space evaluation) and asynchronous (a uniformly sampled set of substates) versions of the OPI scheme with finite policy evaluation rollout converge to the optimal value function and an optimal joint policy asymptotically. Simulation results on a multi-agent MDP with KL control cost variant of the Stag-Hare game validates our scheme's performance in terms of minimizing the cost return.

cs.AI↗

arXiv2023

Learning graph-Fourier spectra of textured surface images for defect localization

In the realm of industrial manufacturing, product inspection remains a significant bottleneck, with only a small fraction of manufactured items undergoing inspection for surface defects. Advances in imaging systems and AI can allow automated full inspection of manufactured surfaces. However, even the most contemporary imaging and machine learning methods perform poorly for detecting defects in images with highly textured backgrounds, that stem from diverse manufacturing processes. This paper introduces an approach based on graph Fourier analysis to automatically identify defective images, as well as crucial graph Fourier coefficients that inform the defects in images amidst highly textured backgrounds. The approach capitalizes on the ability of graph representations to capture the complex dynamics inherent in high-dimensional data, preserving crucial locality properties in a lower dimensional space. A convolutional neural network model (1D-CNN) was trained with the coefficients of the graph Fourier transform of the images as the input to identify, with classification accuracy of 99.4%, if the image contains a defect. An explainable AI method using SHAP (SHapley Additive exPlanations) was used to further analyze the trained 1D-CNN model to discern important spectral coefficients for each image. This approach sheds light on the crucial contribution of low-frequency graph eigen waveforms to precisely localize surface defects in images, thereby advancing the realization of zero-defect manufacturing.

eess.IV↗

arXiv2023

Robust Social Welfare Maximization via Information Design in Linear-Quadratic-Gaussian Games

Information design in an incomplete information game includes a designer with the goal of influencing players' actions through signals generated from a designed probability distribution so that its objective function is optimized. We consider a setting in which the designer has partial knowledge on agents' utilities. We address the uncertainty about players' preferences by formulating a robust information design problem against worst case payoffs. If the players have quadratic payoffs that depend on the players' actions and an unknown payoff-relevant state, and signals on the state that follow a Gaussian distribution conditional on the state realization, then the information design problem under quadratic design objectives is a semidefinite program (SDP). Specifically, we consider ellipsoid perturbations over payoff coefficients in linear-quadratic-Gaussian (LQG) games. We show that this leads to a tractable robust SDP formulation. Numerical studies are carried out to identify the relation between the perturbation levels and the optimal information structures.

math.OC↗

arXiv2023

Maximizing Social Welfare and Agreement via Information Design in Linear-Quadratic-Gaussian Games

We consider linear-quadratic Gaussian (LQG) games in which players have quadratic payoffs that depend on the players' actions and an unknown payoff-relevant state, and signals on the state that follow a Gaussian distribution conditional on the state realization. An information designer decides the fidelity of information revealed to the players in order to maximize the social welfare of the players or reduce the disagreement among players' actions. Leveraging the semi-definiteness of the information design problem, we derive analytical solutions for these objectives under specific LQG games. We show that full information disclosure maximizes social welfare when there is a common payoff-relevant state, when there is strategic substitutability in the actions of players, or when the signals are public. Numerical results show that as strategic substitution increases, the value of the information disclosure increases. When the objective is to induce conformity among players' actions, hiding information is optimal. Lastly, we consider the information design objective that is a weighted combination of social welfare and cohesiveness of players' actions. We obtain an interval for the weights where full information disclosure is optimal under public signals for games with strategic substitutability. Numerical solutions show that the actual interval where full information disclosure is optimal gets close to the analytical interval obtained as substitution increases.

math.OC↗

arXiv2022

Average submodularity of maximizing anticoordination in network games

We consider the control of decentralized learning dynamics for agents in an anti-coordination network game. In the anti-coordination network game, there is a preferred action in the absence of neighbors' actions, and the utility an agent receives from the preferred action decreases as more of its neighbors select the preferred action, potentially causing the agent to select a less desirable action. The decentralized dynamics that is based on the iterated elimination of dominated strategies converge for the considered game. Given a convergent action profile, we measure anti-coordination by the number of edges in the underlying graph that have at least one agent in either end of the edge not taking the preferred action. The maximum anti-coordination (MAC) problem seeks to find an optimal set of agents to control under a finite budget so that the overall network disconnect is maximized on game convergence as a result of the dynamics. We show that the MAC is submodular in expectation in dense bipartite networks for any realization of the utility constants in the population. Utilizing this result, we obtain a performance guarantee for the greedy agent selection algorithm for MAC. Finally, we provide a computational study to show the effectiveness of greedy node selection strategies to solve MAC on general bipartite networks.

math.OC↗

arXiv2022

Information Preferences of Individual Agents in Linear-Quadratic-Gaussian Network Games

We consider linear-quadratic-Gaussian (LQG) network games in which agents have quadratic payoffs that depend on their individual and neighbors' actions, and an unknown payoff-relevant state. An information designer determines the fidelity of information revealed to the agents about the payoff state to maximize the social welfare. Prior results show that full information disclosure is optimal under certain assumptions on the payoffs, i.e., it is beneficial for the average individual. In this paper, we provide conditions based on the strength of the dependence of payoffs on neighbors' actions, i.e., competition, under which a rational agent is expected to benefit, i.e., receive higher payoffs, from full information disclosure. We find that all agents benefit from information disclosure for the star network structure when the game is symmetric and submodular or supermodular. We also identify that the central agent benefits more than a peripheral agent from full information disclosure unless the competition is strong and the number of peripheral agents is small enough. Despite the fact that all agents expect to benefit from information disclosure ex-ante, a central agent can be worse-off from information disclosure in many realizations of the payoff state under strong competition, indicating that a risk-averse central agent can prefer uninformative signals ex-ante.

cs.GT↗

arXiv2022

A Market Mechanism for Trading Flexibility Between Interconnected Electricity Markets

Electricity markets differ in their ability to meet power imbalances in short notice in a controlled fashion. Relatively flexible markets have the ability to ramp up (or down) power flows across interties without compromising their ability to reliably meet internal demand. In this paper, a market mechanism to enable flexibility trading amongst market operators is introduced. In the proposed market mechanism, market operators exchange information regarding optimal terms of trade (nodal prices and flows) along interconnection lines at every trading round. Equipped with this information, each market operator then independently solves its own internal chance-constrained economic dispatch problem and broadcasts the updated optimal terms of trade for flows across markets. We show the proposed decentralized market mechanism for flexibility trading converges to a Nash equilibrium of the intraday market coupling game, i.e. a combination of internal market clearing solutions (one for each participating market) and flows and prices along interconnection lines so that no individual market operator has an incentive to modify its own internal solution and/or the terms of trade along interties. For a specific class of chance constraints, we show that the limiting equilibrium outcome is efficient, i.e. it corresponds to the solution of the single market clearing problem for all participating markets. The proposed market mechanism is illustrated with an application to the three-area IEEE Reliability Test System.

cs.GT↗

arXiv2022

Decentralized Fictitious Play Converges Near a Nash Equilibrium in Near-Potential Games

We investigate convergence of decentralized fictitious play (DFP) in near-potential games, wherein agents preferences can almost be captured by a potential function. In DFP agents keep local estimates of other agents' empirical frequencies, best-respond against these estimates, and receive information over a time-varying communication network. We prove that empirical frequencies of actions generated by DFP converge around a single Nash Equilibrium (NE) assuming that there are only finitely many Nash equilibria, and the difference in utility functions resulting from unilateral deviations is close enough to the difference in the potential function values. This result assures that DFP has the same convergence properties of standard Fictitious play (FP) in near-potential games.

cs.GT↗

arXiv2021

A Best-Response Algorithm with Voluntary Communication and Mobility Protocols for Mobile Autonomous Teams Solving the Target Assignment Problem

We consider a team of mobile autonomous robots with the aim to cover a given set of targets. Each robot aims to select a target to cover and physically reach it by the final time in coordination with other robots given the locations of targets. Robots are unaware of which targets other robots intend to cover. Each robot can control its mobility and who to send information to. We assume communication happens over a wireless channel that is subject to fading and failures. Given the setup, we propose a decentralized algorithm based on decentralized fictitious play in which robots reason about the selections and locations of other robots to decide which target to select, whether to communicate or not, who to communicate with, and where to move. Specifically, the communication actions of the robots are learning-aware, and their mobility actions are sensitive to the success probability of communication. We show that the decentralized algorithm guarantees that robots will cover their targets in finite time. Numerical simulations and experiments using a team of mobile robots confirm the target coverage in finite time and show that mobility control for communication and learning-aware voluntary communication protocols reduce the number of communication attempts in comparison to a benchmark distributed algorithm that relies on communication after every decision epoch.

eess.SY↗

arXiv2021

Decentralized Inertial Best-Response with Voluntary and Limited Communication in Random Communication Networks

Multiple autonomous agents interact over a random communication network to maximize their individual utility functions which depend on the actions of other agents. We consider decentralized best-response with inertia type algorithms in which agents form beliefs about the future actions of other players based on local information, and take an action that maximizes their expected utility computed with respect to these beliefs or continue to take their previous action. We show convergence of these types of algorithms to a Nash equilibrium in weakly acyclic games under the condition that the belief update and information exchange protocols successfully learn the actions of other players with positive probability in finite time given a static environment, i.e., when other agents' actions do not change. We design a decentralized fictitious play algorithm with voluntary and limited communication (DFP-VL) protocols that satisfy this condition. In the voluntary communication protocol, each agent decides whom to exchange information with by assessing the novelty of its information and the potential effect of its information on others' assessments of their utility functions. The limited communication protocol entails agents sending only their most frequent action to agents that they decide to communicate with. Numerical experiments on a target assignment game demonstrate that the voluntary and limited communication protocol can more than halve the number of communication attempts while retaining the same convergence rate as DFP in which agents constantly attempt to communicate.

math.OC↗

arXiv2021

Decentralized Fictitious Play in Near-Potential Games with Time-Varying Communication Networks

We study the convergence properties of decentralized fictitious play (DFP) for the class of near-potential games where the incentives of agents are nearly aligned with a potential function. In DFP, agents share information only with their current neighbors in a sequence of time-varying networks, keep estimates of other agents' empirical frequencies, and take actions to maximize their expected utility functions computed with respect to the estimated empirical frequencies. We show that empirical frequencies of actions converge to a set of strategies with potential function values that are larger than the potential function values obtained by approximate Nash equilibria of the closest potential game. This result establishes that DFP has identical convergence guarantees in near-potential games as the standard fictitious play in which agents observe the past actions of all the other agents.

math.OC↗

arXiv2021

Distributed Networked Learning with Correlated Data

We consider a distributed estimation method in a setting with heterogeneous streams of correlated data distributed across nodes in a network. In the considered approach, linear models are estimated locally (i.e., with only local data) subject to a network regularization term that penalizes a local model that differs from neighboring models. We analyze computation dynamics (associated with stochastic gradient updates) and information exchange (associated with exchanging current models with neighboring nodes). We provide a finite-time characterization of convergence of the weighted ensemble average estimate and compare this result to federated learning, an alternative approach to estimation wherein a single model is updated by locally generated gradient updates. This comparison highlights the trade-off between speed vs precision: while model updates take place at a faster rate in federated learning, the proposed networked approach to estimation enables the identification of models with higher precision. We illustrate the method's general applicability in two examples: estimating a Markov random field using wireless sensor networks and modeling prey escape behavior of flocking birds based on a publicly available dataset.

cs.LG↗

arXiv2021

An Iterative Mechanism for Coupling Electricity Markets

The coordinated operation of interconnected but locally controlled electricity markets is generally referred to as a "coupling". In this paper we propose a new mechanism design for efficient coupling of independent electricity markets. The mechanism operates after each individual market has settled (e.g. hour-ahead) and based upon the reported supply and demand functions for internal market optimization (clearing), each market operator is asked to iteratively quote the terms of energy trade (on behalf of the agents participating in its market) across the transmission lines connecting to other markets. The mechanism is scalable as the informational demands placed on each market operator at each iteration are limited. We show that the mechanism's outcome converges to the optimal flows between markets given the reported supply and demand functions from each individual market clearing. We show the proposed market coupling design does not alter the structure of incentives in each internal market, i.e., any internal market equilibrium will remain so (approximately) after coupling is implemented. This is achieved via incentive transfers (updated at each iteration) that remunerate each market with its marginal contribution (i.e. cost savings) to all other participating markets. We identify a sufficient condition on a uniform participation fee for each market operator ensuring the mechanism incurs no deficit. The proposed decentralized mechanism is implemented on the three-area IEEE Reliability Test System where the simulation results showcase the efficiency of proposed model.

math.OC↗

arXiv2021

Optimal evolutionary control for artificial selection on molecular phenotypes

Controlling an evolving population is an important task in modern molecular genetics, including directed evolution for improving the activity of molecules and enzymes, in breeding experiments in animals and in plants, and in devising public health strategies to suppress evolving pathogens. An optimal intervention to direct evolution should be designed by considering its impact over an entire stochastic evolutionary trajectory that follows. As a result, a seemingly suboptimal intervention at a given time can be globally optimal as it can open opportunities for desirable actions in the future. Here, we propose a feedback control formalism to devise globally optimal artificial selection protocol to direct the evolution of molecular phenotypes. We show that artificial selection should be designed to counter evolutionary tradeoffs among multi-variate phenotypes to avoid undesirable outcomes in one phenotype by imposing selection on another. Control by artificial selection is challenged by our ability to predict molecular evolution. We develop an information theoretical framework and show that molecular time-scales for evolution under natural selection can inform how to monitor a population in order to acquire sufficient predictive information for an effective intervention with artificial selection. Our formalism opens a new avenue for devising artificial selection methods for directed evolution of molecular functions.

q-bio.PE↗

arXiv2019

Distributed Fictitious Play in Potential Games with Time-Varying Communication Networks

We propose a distributed algorithm for multiagent systems that aim to optimize a common objective when agents differ in their estimates of the objective-relevant state of the environment. Each agent keeps an estimate of the environment and a model of the behavior of other agents. The model of other agents' behavior assumes agents choose their actions randomly based on a stationary distribution determined by the empirical frequencies of past actions. At each step, each agent takes the action that maximizes its expectation of the common objective computed with respect to its estimate of the environment and its model of others. We propose a weighted averaging rule with non-doubly stochastic weights for agents to estimate the empirical frequency of past actions of all other agents by exchanging their estimates with their neighbors over a time-varying communication network. Under this averaging rule, we show agents' estimates converge to the actual empirical frequencies fast enough. This implies convergence of actions to a Nash equilibrium of the game with identical payoffs given by the expectation of the common objective with respect to an asymptotically agreed estimate of the state of the environment.

eess.SY↗

arXiv2018

Control of learning in anti-coordination network games

We consider control of heterogeneous players repeatedly playing an anti-coordination network game. In an anti-coordination game, each player has an incentive to differentiate its action from its neighbors. At each round of play, players take actions according to a learning algorithm that mimics the iterated elimination of strictly dominated strategies. We show that the learning dynamics may fail to reach anti-coordination in certain scenarios. We formulate an optimization problem with the objective to reach maximum anti-coordination while minimizing the number of players to control. We consider both static and dynamic control policy formulations. Relating the problem to a minimum vertex cover problem on bipartite networks, we develop a feasible dynamic policy that is efficient to compute. Solving for optimal policies on benchmark networks show that the vertex cover based policy can be a loose upper bound when there is a potential to make use of cascades caused by the learning dynamics of uncontrolled players. We propose an algorithm that finds feasible, though possibly suboptimal, policies by sequentially adding players to control considering their cascade potential. Numerical experiments on random networks show the cascade-based algorithm can lower the control effort significantly compared to simpler control schemes.

eess.SY↗