arXiv subjects

Yoav Kolumbus

Publications and source records attributed to Yoav Kolumbus.

16 recordsLinked to original sources

arXiv2026

Robust Temporal Guarantees in Budgeted Sequential Auctions

In modern advertising platforms, learning algorithms are deployed by budget-constrained bidders to maximize their accumulated value. These algorithms often offer classical utility guarantees like no-regret, i.e., the agent's utility is at least the utility achieved by some benchmark in which it is assumed that every other agent's bidding remains the same. These guarantees offer compelling properties: They are optimal against stationary competition distributions, and in unconstrained settings, the resulting empirical distribution of play induced by no-regret dynamics approximates a Coarse Correlated Equilibrium. However, no-regret algorithms are easily manipulable, and in budgeted settings, no stronger notion of regret (such as swap regret) is currently known that would limit such manipulation. We propose a very simple learning algorithm for budgeted sequential auctions where agents maximize their total number of wins and show that it has surprisingly appealing properties. We analyze this algorithm from two perspectives. First, we show that when an agent with a $\rho$ fraction of the total budget uses this algorithm, then she is guaranteed to win at least $\rho T - O(\sqrt T)$ of the total $T$ rounds. This result holds for adversarial behavior by the other agents, as long as they respect their own budget restrictions. Second, we examine the scenario when all the agents follow our algorithm. By the first result, every agent's total wins are proportional to her budget, up to the additive $O(\sqrt T)$ term. In addition, we show that this result holds in a much stronger sense: after an initial period of $O(\sqrt T \log T)$ rounds, every agent gets the same guarantee over any time interval. For intervals of length $O(\sqrt T)$, we show that the deviation from the desired number of wins is an additive constant.

cs.GT↗

arXiv2025

Learning in Strategic Queuing Systems with Small Buffers

We consider learning outcomes in games with carryover effects between rounds: when outcomes in the present round affect the game in the future. An important example of such systems is routers in networking, as they use simple learning algorithms to find the best way to deliver packets to their desired destination. This simple, myopic, and distributed decision process makes large queuing systems easy to operate, but at the same time, the system needs more capacity than would be required if all traffic were centrally coordinated. Gaitonde and Tardos (EC 2020 and JACM 2023) initiated the study of such systems, modeling them as an infinitely repeated game in which routers compete for servers and the system maintains a state (the number of packets held at each queue) that results from outcomes of previous rounds. However, their model assumes that servers have no buffers at all, so routers have to resend all packets that were not served successfully, which makes their system model unrealistic. They show that in their model, even with hugely increased server capacity relative to what is needed in the centrally coordinated case, ensuring that the system is stable requires the use of timestamps and priority for older packets. We consider a system with two important changes, which make the model more realistic and allow for much higher traffic rates: first, we add a very small buffer to each server, allowing the server to hold on to a single packet to be served later (if it fails to serve it immediately), and second, we do not require timestamps or priority to older packets. Using theoretical analysis and simulations, we show that when queues are learning, a small constant-factor increase in server capacity, compared to what would be needed if centrally coordinating, suffices to keep the system stable, even if servers select randomly among packets arriving simultaneously.

cs.GT↗

arXiv2025

Markets with Heterogeneous Agents: Dynamics and Survival of Bayesian vs. No-Regret Learners

We analyze the performance of heterogeneous learning agents in asset markets with stochastic payoffs. Our main focus is on comparing Bayesian learners and no-regret learners who compete in markets and identifying the conditions under which each approach is more effective. We formally relate the notions of survival and market dominance studied in economics and the framework of regret minimization, thereby bridging these theories. A central finding is that regret plays a key role in market selection, but low regret alone does not guarantee survival: surprisingly, an agent may achieve even logarithmic regret and yet be driven out of the market when competing against a Bayesian learner with a finite prior that assigns positive probability to the correct model. At the same time, we show that Bayesian learning is highly fragile, while no-regret learning requires less knowledge of the environment and is therefore more robust. Motivated by this contrast, we propose two simple hybrid strategies that incorporate Bayesian updates while improving robustness and adaptability to distribution shifts, taking a step toward a best-of-both-worlds learning approach. More broadly, our work contributes to the understanding of dynamics of heterogeneous learning agents and their impact on markets.

cs.GT↗

arXiv2024

Learning in Budgeted Auctions with Spacing Objectives

In many repeated auction settings, participants care not only about how frequently they win but also how their winnings are distributed over time. This problem arises in various practical domains where avoiding congested demand is crucial, such as online retail sales and compute services, as well as in advertising campaigns that require sustained visibility over time. We introduce a simple model of this phenomenon, modeling it as a budgeted auction where the value of a win is a concave function of the time since the last win. This implies that for a given number of wins, even spacing over time is optimal. We also extend our model and results to the case when not all wins result in "conversions" (realization of actual gains), and the probability of conversion depends on a context. The goal is to maximize and evenly space conversions rather than just wins. We study the optimal policies for this setting in second-price auctions and offer learning algorithms for the bidders that achieve low regret against the optimal bidding policy in a Bayesian online setting. Our main result is a computationally efficient online learning algorithm that achieves $\tilde O(\sqrt T)$ regret. We achieve this by showing that an infinite-horizon Markov decision process (MDP) with the budget constraint in expectation is essentially equivalent to our problem, even when limiting that MDP to a very small number of states. The algorithm achieves low regret by learning a bidding policy that chooses bids as a function of the context and the system's state, which will be the time elapsed since the last win (or conversion). We show that state-independent strategies incur linear regret even without uncertainty of conversions. We complement this by showing that there are state-independent strategies that, while still having linear regret, achieve a $(1-\frac 1 e)$ approximation to the optimal reward.

cs.GT↗

arXiv2024

Games with Payments between Learning Agents

In repeated games, such as auctions, players rely on autonomous learning agents to choose their actions. We study settings in which players have their agents make monetary transfers to other agents during play at their own expense, in order to influence learning dynamics in their favor. Our goal is to understand when players have incentives to use such payments, how payments between agents affect learning outcomes, and what the resulting implications are for welfare and its distribution. We propose a simple game-theoretic model to capture the incentive structure of such scenarios. We find that, quite generally, abstaining from payments is not robust to strategic deviations by users of learning agents: self-interested players benefit from having their agents make payments to other learners. In a broad class of games, such endogenous payments between learning agents lead to higher welfare for all players. In first- and second-price auctions, equilibria of the induced "payment-policy game" lead to highly collusive learning outcomes, with low or vanishing revenue for the auctioneer. These results highlight a fundamental challenge for mechanism design, as well as for regulatory policies, in environments where learning agents may interact in the digital ecosystem beyond a mechanism's boundaries.

cs.GT↗

arXiv2024

Contracting with a Learning Agent

Many real-life contractual relations differ completely from the clean, static model at the heart of principal-agent theory. Typically, they involve repeated strategic interactions of the principal and agent, taking place under uncertainty and over time. While appealing in theory, players seldom use complex dynamic strategies in practice, often preferring to circumvent complexity and approach uncertainty through learning. We initiate the study of repeated contracts with a learning agent, focusing on agents who achieve no-regret outcomes. Optimizing against a no-regret agent is a known open problem in general games; we achieve an optimal solution to this problem for a canonical contract setting, in which the agent's choice among multiple actions leads to success/failure. The solution has a surprisingly simple structure: for some $α> 0$, initially offer the agent a linear contract with scalar $α$, then switch to offering a linear contract with scalar $0$. This switch causes the agent to ``free-fall'' through their action space and during this time provides the principal with non-zero reward at zero cost. Despite apparent exploitation of the agent, this dynamic contract can leave \emph{both} players better off compared to the best static contract. Our results generalize beyond success/failure, to arbitrary non-linear contracts which the principal rescales dynamically. Finally, we quantify the dependence of our results on knowledge of the time horizon, and are the first to address this consideration in the study of strategizing against learning agents.

cs.GT↗

arXiv2024

Asynchronous Proportional Response Dynamics in Markets with Adversarial Scheduling

We study Proportional Response Dynamics (PRD) in linear Fisher markets where participants act asynchronously. We model this scenario as a sequential process in which in every step, an adversary selects a subset of the players that will update their bids, subject to liveness constraints. We show that if every bidder individually uses the PRD update rule whenever they are included in the group of bidders selected by the adversary, then (in the generic case) the entire dynamic converges to a competitive equilibrium of the market. Our proof technique uncovers further properties of linear Fisher markets, such as the uniqueness of the equilibrium for generic parameters and the convergence of associated best-response dynamics and no-swap regret dynamics under certain conditions.

cs.GT↗

arXiv2023

Game Manipulators -- the Strategic Implications of Binding Contracts

Commitment devices are powerful tools that can influence and incentivise certain behaviours by linking them to rewards or punishments. These devices are particularly useful in decision-making, as they can steer individuals towards specific choices. In the field of game theory, commitment devices can alter a player's payoff matrix, ultimately changing the game's Nash equilibria. Interestingly, agents, whom we term game manipulators and who can be external to the original game, can leverage such devices to extract fees from players by making them contingent offers that modify the payoffs of their actions. This can result in a different Nash equilibrium with potentially lower payoffs for the players compared to the original game. For this scheme to work, it is required that all commitments be binding, meaning that once an offer is made, it cannot be revoked. Consequently, we analyse binding contracts as the commitment mechanism that enables game manipulation scenarios. The main focus of this study is to formulate the logic of this setting, expand its scope to encompass more intricate schemes, and analyse the behaviour of regret-minimizing agents in scenarios involving game manipulation.

cs.GT↗

arXiv2023

How and Why to Manipulate Your Own Agent: On the Incentives of Users of Learning Agents

The usage of automated learning agents is becoming increasingly prevalent in many online economic applications such as online auctions and automated trading. Motivated by such applications, this paper is dedicated to fundamental modeling and analysis of the strategic situations that the users of automated learning agents are facing. We consider strategic settings where several users engage in a repeated online interaction, assisted by regret-minimizing learning agents that repeatedly play a "game" on their behalf. We propose to view the outcomes of the agents' dynamics as inducing a "meta-game" between the users. Our main focus is on whether users can benefit in this meta-game from "manipulating" their own agents by misreporting their parameters to them. We define a general framework to model and analyze these strategic interactions between users of learning agents for general games and analyze the equilibria induced between the users in three classes of games. We show that, generally, users have incentives to misreport their parameters to their own agents, and that such strategic user behavior can lead to very different outcomes than those anticipated by standard analysis.

cs.GT↗

arXiv2022

Explainable Reinforcement Learning via Model Transforms

Understanding emerging behaviors of reinforcement learning (RL) agents may be difficult since such agents are often trained in complex environments using highly complex decision making procedures. This has given rise to a variety of approaches to explainability in RL that aim to reconcile discrepancies that may arise between the behavior of an agent and the behavior that is anticipated by an observer. Most recent approaches have relied either on domain knowledge that may not always be available, on an analysis of the agent's policy, or on an analysis of specific elements of the underlying environment, typically modeled as a Markov Decision Process (MDP). Our key claim is that even if the underlying model is not fully known (e.g., the transition probabilities have not been accurately learned) or is not maintained by the agent (i.e., when using model-free methods), the model can nevertheless be exploited to automatically generate explanations. For this purpose, we suggest using formal MDP abstractions and transforms, previously used in the literature for expediting the search for optimal policies, to automatically produce explanations. Since such transforms are typically based on a symbolic representation of the environment, they can provide meaningful explanations for gaps between the anticipated and actual agent behavior. We formally define the explainability problem, suggest a class of transforms that can be used for explaining emergent behaviors, and suggest methods that enable efficient search for an explanation. We demonstrate the approach on a set of standard benchmarks.

cs.AI↗

arXiv2022

Auctions Between Regret-Minimizing Agents

We analyze a scenario in which software agents implemented as regret-minimizing algorithms engage in a repeated auction on behalf of their users. We study first-price and second-price auctions, as well as their generalized versions (e.g., as those used for ad auctions). Using both theoretical analysis and simulations, we show that, surprisingly, in second-price auctions the players have incentives to misreport their true valuations to their own learning agents, while in the first-price auction it is a dominant strategy for all players to truthfully report their valuations to their agents.

cs.GT↗

arXiv2022

Optimal Collaterals in Multi-Enterprise Investment Networks

We study a market of investments on networks, where each agent (vertex) can invest in any enterprise linked to her, and at the same time, raise capital for her firm's enterprise from other agents she is linked to. Failing to raise sufficient capital results with the firm defaulting, being unable to invest in others. Our main objective is to examine the role of collateral contracts in handling the strategic risk that can propagate to a systemic risk throughout the network in a cascade of defaults. We take a mechanism-design approach and solve for the optimal scheme of collateral contracts that capital raisers offer their investors. These contracts aim at sustaining the efficient level of investment as a unique Nash equilibrium, while minimizing the total collateral. Our main results contrast the network environment with its non-network counterpart (where the sets of investors and capital raisers are disjoint). We show that for acyclic investment networks, the network environment does not necessitate any additional collaterals, and systemic risk can be fully handled by optimal bilateral collateral contracts between capital raisers and their investors. This is, unfortunately, not the case for cyclic investment networks. We show that bilateral contracting will not suffice to resolve systemic risk, and the market will need an external entity to design a global collateral scheme for all capital raisers. Furthermore, the minimum total collateral that will sustain the efficient level of investment as a unique equilibrium may be arbitrarily higher, even in simple cyclic investment networks, compared with its corresponding non-network environment. Additionally, we prove computational-complexity results, both for a single enterprise and for networks.

cs.GT↗

arXiv2020

On the Effectiveness of Tracking and Testing in SEIR Models

We study the effectiveness of tracking and testing in mitigating or suppressing epidemic outbreaks, in combination with or as an alternative to quarantines and global lockdowns. We study these intervention methods on a network-based SEIR model, augmented with an additional probability to model symptomatic, asymptomatic and pre-symptomatic cases. Our focus is on the basic trade-offs between economic costs and human lives lost, and how these trade-offs change under different lockdown, quarantine, tracking and testing policies. Our main findings are as follows: (i) Tests combined with patient quarantines reduce both economic costs and mortality, but require a large-scale testing capacity to achieve a significant improvement; (ii) Tracking significantly reduces both economic costs and mortality; (iii) Tracking combined with a limited number of tests can achieve containment without lockdowns; (iv) If there is a small flow of new incoming infections, dynamic "On-Off" lockdowns are more efficient than fixed lockdowns. Our simulation results underline the extreme effectiveness of tracking and testing policies in reducing both economic costs and mortality and their potential to contain epidemic outbreaks without imposing social distancing restrictions. This highlights the difficult social question of trading-off these gains with the privacy loss that tracking necessarily entails.

q-bio.PE↗

arXiv2019

Neural Networks for Predicting Human Interactions in Repeated Games

We consider the problem of predicting human players' actions in repeated strategic interactions. Our goal is to predict the dynamic step-by-step behavior of individual players in previously unseen games. We study the ability of neural networks to perform such predictions and the information that they require. We show on a dataset of normal-form games from experiments with human participants that standard neural networks are able to learn functions that provide more accurate predictions of the players' actions than established models from behavioral economics. The networks outperform the other models in terms of prediction accuracy and cross-entropy, and yield higher economic value. We show that if the available input is only of a short sequence of play, economic information about the game is important for predicting behavior of human agents. However, interestingly, we find that when the networks are trained with long enough sequences of history of play, action-based networks do well and additional economic details about the game do not improve their performance, indicating that the sequence of actions encode sufficient information for the success in the prediction task.

cs.GT↗

arXiv2018

Phase Transition in the Maximal Influence Problem: When Do We Need Optimization?

Considerable efforts were made in recent years in devising optimization algorithms for influence maximization in networks. Here we ask: "When do we need optimization?" We use results from statistical mechanics and direct simulations on ER networks, small-world networks, power-law networks and a dataset of real-world networks to characterize the parameter-space region where optimization is required. We show that in both synthetic and real-world networks this optimization region is due to a well known physical phase transition of the network, and that it vanishes as a power-law with the network size. We then show that also from a utility-maximization perspective (when considering the costs of the optimization process), for large networks standard optimization is profitable only in a vanishing parameter region near the phase transition. Finally, we introduce a novel constant-time optimization approach, and demonstrate it through a simple algorithm that manages to give similar results to standard optimization methods in terms of the influenced-set size, while improving the results in terms of the net utility.

cs.SI↗

arXiv2016

Behavior-Based Machine-Learning: A Hybrid Approach for Predicting Human Decision Making

A large body of work in behavioral fields attempts to develop models that describe the way people, as opposed to rational agents, make decisions. A recent Choice Prediction Competition (2015) challenged researchers to suggest a model that captures 14 classic choice biases and can predict human decisions under risk and ambiguity. The competition focused on simple decision problems, in which human subjects were asked to repeatedly choose between two gamble options. In this paper we present our approach for predicting human decision behavior: we suggest to use machine learning algorithms with features that are based on well-established behavioral theories. The basic idea is that these psychological features are essential for the representation of the data and are important for the success of the learning process. We implement a vanilla model in which we train SVM models using behavioral features that rely on the psychological properties underlying the competition baseline model. We show that this basic model captures the 14 choice biases and outperforms all the other learning-based models in the competition. The preliminary results suggest that such hybrid models can significantly improve the prediction of human decision making, and are a promising direction for future research.

cs.LG↗