SearcharxivSearch

arXiv subjects

Moshe Tennenholtz

Publications and source records attributed to Moshe Tennenholtz.

At least 19 recordsLinked to original sources

Learning Dynamics of Strategic Publishers in Generative AI Ecosystems

Generative AI (GenAI) search systems are transforming how users access information. Unlike ranking-based search systems, where users observe a ranked list of documents, GenAI search systems, given a user's question, generate an answer, often accompanied by external sources (e.g., in the form of citations). Content creators (publishers) seeking to increase exposure might behave strategically and compete with other creators for users' attention. While publishers in ranking-based systems might strategically modify their content to improve its ranking, the incentives in generative systems take on a new form. Publishers may now gain exposure through generated responses and attributions to those responses. We introduce a novel game-theoretic model of the emerging GenAI ecosystem in which publishers compete for attribution-based exposure. We study the learning dynamics of strategic content creators under better-response dynamics. We associate the convergence of learning dynamics to equilibrium with ecosystem stability. Employing the notion of potential games, we study the stability of GenAI ecosystems under several known content selection mechanisms. We demonstrate the instability of mechanisms representing real-world modern systems and characterize a mechanism that induces a stable ecosystem. We conduct extensive simulations to analyze the stability and welfare of GenAI ecosystems under various mechanisms. The simulations support our theoretical findings and reveal an interplay among stability, publisher welfare, and user welfare. In particular, stable mechanisms do not necessarily maximize welfare, demonstrating an important trade-off for platform designers. We then introduce a study illustrating that the proper selection of the GenAI mechanism enables the manifestation of desired trade-offs between publisher welfare and the different sources of user welfare.

cs.GT

Stability in Competitive Search with Results Diversification

In a competitive search setting, publishers strategically modify their documents in response to induced rankings so as to improve their future ranking. We present a novel game-theoretic analysis of a competitive search setting where search-results diversification is applied. Our analysis reveals an inherent tradeoff between corpus diversity and corpus stability, where the latter corresponds to an equilibrium in a game. We analyze two representative diversification methods and show that stability need not necessarily be reached, leaving the corpus to rapid changes due to ranking incentivized modifications of publishers. We then present a novel approach to devise diversification-based ranking functions that are guaranteed to lead to corpus stability.

cs.GT

Predicting Decisions of AI Agents from Limited Interaction through Text-Tabular Modeling

AI agents negotiate and transact in natural language with unfamiliar counterparts: a buyer bot facing an unknown seller, or a procurement assistant negotiating with a supplier. In such interactions, the counterpart's LLM, prompts, control logic, and rule-based fallbacks are hidden, while each decision can have monetary consequences. We ask whether an agent can predict an unfamiliar counterpart's next decision from a few interactions. To avoid real-world logging confounds, we study this problem in controlled bargaining and negotiation games, formulating it as target-adaptive text-tabular prediction: each decision point is a table row combining structured game state, offer history, and dialogue, while $K$ previous games of the same target agent, i.e., the counterpart being modeled, are provided in the prompt as labeled adaptation examples. Our model is built on a tabular foundation model that represents rows using game-state features and LLM-based text representations, and adds LLM-as-Observer as an additional representation: a small frozen LLM reads the decision-time state and dialogue; its answer is discarded, and its hidden state becomes a decision-oriented feature, making the LLM an encoder rather than a direct few-shot predictor. Training on 13 frontier-LLM agents and testing on 91 held-out scaffolded agents, the full model outperforms direct LLM-as-Predictor prompting and game+text features baselines. Within this tabular model, Observer features contribute beyond the other feature schemes: at $K=16$, they improve response-prediction AUC by about 4 points across both tasks and reduce bargaining offer-prediction error by 14%. These results show that formulating counterpart prediction as a target-adaptive text-tabular task enables effective adaptation, and that hidden LLM representations expose decision-relevant signals that direct prompting does not surface.

cs.LG

Alignment Makes Language Models Normative, Not Descriptive

Post-training alignment optimizes language models to match human preference signals, but this objective is not equivalent to modeling observed human behavior. We compare 120 base-aligned model pairs on more than 10,000 real human decisions in multi-round strategic games - bargaining, persuasion, negotiation, and repeated matrix games. In these settings, base models outperform their aligned counterparts in predicting human choices by nearly 10:1, robustly across model families, prompt formulations, and game configurations. This pattern reverses, however, in settings where human behavior is more likely to follow normative predictions: aligned models dominate on one-shot textbook games across all 12 types tested and on non-strategic lottery choices - and even within the multi-round games themselves, at round one, before interaction history develops. This boundary-condition pattern suggests that alignment induces a normative bias: it improves prediction when human behavior is relatively well captured by normative solutions, but hurts prediction in multi-round strategic settings, where behavior is shaped by descriptive dynamics such as reciprocity, retaliation, and history-dependent adaptation. These results reveal a fundamental trade-off between optimizing models for human use and using them as proxies for human behavior.

cs.CL

Addressing Corpus Knowledge Poisoning Attacks on RAG Using Sparse Attention

Retrieval Augmented Generation (RAG) is a highly effective paradigm for keeping LLM-based responses up-to-date and reducing the likelihood of hallucinations. Yet, RAG was recently shown to be quite vulnerable to corpus knowledge poisoning: an attacker injects misleading documents to the corpus to steer an LLM's output to an undesired response. We argue that the standard causal attention mechanism in LLMs enables harmful cross-document interactions, specifically in cases of attacks. Accordingly, we introduce a novel defense approach for RAG: Sparse Document Attention RAG (SDAG). This is a block-sparse attention mechanism that disallows cross-attention between retrieved documents. SDAG requires a minimal inference-time change to the attention mask. We present an empirical evaluation of LLM-based question answering (QA) with a variety of attack strategies on RAG. We show that our SDAG method substantially outperforms the standard causal attention mechanism. We further demonstrate the clear merits of integrating SDAG with state-of-the-art RAG defense methods. Specifically, the integration results in performance that is statistically significantly better than the state-of-the-art.

cs.IR

Sequential LLM Release Facilitates Manipulation in Regulated Markets

AI agents increasingly mediate bargaining, negotiation and persuasion for people and firms. Such markets extend software-mediated commerce, but add a governance problem: independent model releases change delegates available to participants. Game theory shows that expanding a strategy set can harm equilibrium outcomes, but mostly through constructed examples. Deployed AI-agent logs are scarce, proprietary and privacy-sensitive, and lack counterfactuals and payoff labels. We therefore use GLEE, an independently collected benchmark of 587K strategic decisions by 13 large language models across 1,320 matched bargaining, negotiation and persuasion configurations, to study model release as strategy expansion. Across more than 50{,}000 release comparisons, many releases move payoffs in opposite directions: one agent gains while the other loses. We identify the Poisoned Apple effect: a released model that no agent adopts in equilibrium nevertheless shifts payoffs in opposite directions and changes the regulator's market design. Up to roughly three in ten opposing shifts arise this way, and technology restrictions can amplify the effect.

cs.GT

Cooperation Under Network-Constrained Communication

In this paper, we study cooperation in distributed games under network-constrained communication. Building on the framework of Monderer and Tennenholtz (1999), we derive a sufficient condition for cooperative equilibrium in settings where communication between agents is delayed by the underlying network topology. Each player deploys an agent at every location, and local interactions follow a Prisoner's Dilemma structure. We derive a sufficient condition that depends on the network diameter and the number of locations, and analyze extreme cases of instantaneous, delayed, and proportionally delayed communication. We also discuss the asymptotic case of scale-free communication networks, in which the network diameter grows sub-linearly in the number of locations. These insights clarify how communication latency and network design jointly determine the emergence of distributed cooperation.

cs.GT

RLRF: Competitive Search Agent Design via Reinforcement Learning from Ranker Feedback

Competitive search is a setting where document publishers modify them to improve their ranking in response to a query. Recently, publishers have increasingly leveraged LLMs to generate and modify competitive content. We introduce Reinforcement Learning from Ranker Feedback (RLRF), a framework that trains LLMs using preference datasets derived from ranking competitions. The goal of a publisher (LLM-based) agent is to optimize content for improved ranking while accounting for the strategies of competing agents. We generate the datasets using approaches that do not rely on human-authored data. We show that our proposed agents consistently and substantially outperform previously suggested approaches for LLM-based competitive document modification. We further show that our agents are effective with ranking functions they were not trained for (i.e., out of distribution) and they adapt to strategic opponents. These findings provide support to the significant potential of using reinforcement learning in competitive search.

cs.IR

AI Behavioral Science

We outline a foundation for a new field of ``AI Behavioral Science,'' covering three perspectives. First, as AI becomes ubiquitous and is increasingly proprietary and opaque, it becomes vital to develop techniques for assessing AI behavior. We outline how tools developed to assess people's behaviors by social scientists can be used to assess and infer AI's behaviors biases, tendencies, and heuristics. Second, we also discuss how AI can change the ways in which we learn about human behavior. Beyond its computational power, AI offers new techniques for simulating, inferring, and predicting human behaviors that we outline and discuss. Third, as humans and AI are interacting in increasingly complex and intertwined systems, we need to understand the implications for the resulting economic and political outcomes. We outline issues that are increasingly pressing concerning the future of human-AI interactions and potential changes and disruptions that can ensue.

cs.HC

When Competition Helps: Achieving Optimal Traffic Flow with Multiple Autonomous Planners

The inefficiency of selfish routing in congested networks is a classical problem in algorithmic game theory, often captured by the Price of Anarchy (i.e., the ratio between the social cost of decentralized decisions and that of a centrally optimized solution.) With the advent of autonomous vehicles, capable of receiving and executing centrally assigned routes, it is natural to ask whether their deployment can eliminate this inefficiency. At first glance, a central authority could simply compute an optimal traffic assignment and instruct each vehicle to follow its assigned path. However, this vision overlooks critical challenges: routes must be individually rational (no vehicle has an incentive to deviate), and in practice, multiple planning agents (e.g., different companies) may coexist and compete. Surprisingly, we show that such competition is not merely an obstacle but a necessary ingredient for achieving optimal outcomes. In this work, we design a routing mechanism that embraces competition and converges to an optimal assignment, starting from the classical Pigou network as a foundational case.

cs.GT

The Search for Stability: Learning Dynamics of Strategic Publishers with Initial Documents

We study a game-theoretic information retrieval model in which strategic publishers aim to maximize their chances of being ranked first by the search engine while maintaining the integrity of their original documents. We show that the commonly used Probability Ranking Principle (PRP) ranking scheme results in an unstable environment where games often fail to reach pure Nash equilibrium. We propose two families of ranking functions that do not adhere to the PRP principle. We provide both theoretical and empirical evidence that these methods lead to a stable search ecosystem, by providing positive results on the learning dynamics convergence. We also define the publishers' and users' welfare, demonstrate a possible publisher-user trade-off, and provide means for a search system designer to control it. Finally, we show how instability harms long-term users' welfare.

cs.GT

Near-Linear MIR Algorithms for Stochastically-Ordered Priors

With the rise of online applications, recommender systems (RSs) often encounter constraints in balancing exploration and exploitation. Such constraints arise when exploration is carried out by agents whose utility must be taken into account when optimizing overall welfare. A recent work by Bahar et al. (2020) suggests that recommendations should be \emph{mechanism-informed individually rational} (MIR). Specifically, if agents have a default arm they would use, relying on the RS should yield each agent at least the reward of the default arm, conditioned on the information available to the RS. Under the MIR constraint, striking a balance between exploration and exploitation becomes a complex planning problem. To that end, Bahar et al. propose an approximately optimal yet inefficient planning algorithm that runs in $O(2^K K^2 H^2)$, where $K$ is the number of arms and $H$ is the size of the support of the reward distributions. In this paper, we make a significant improvement for a special yet practical case, removing both the dependence on $H$ and the exponential dependence on $K$. We assume a stochastic order of the rewards (e.g., Gaussian with unit variance, Bernoulli, etc.), and devise an asymptotically optimal algorithm with a runtime of $O(K \log K)$. Our technique is based on formulating a Goal Markov Decision Process (GMDP), establishing an optimal dynamic programming procedure, and then unveiling its crux -- fleshing out a simple index-based structure that facilitates efficient computation. Additionally, we present an incentive-compatible version of our algorithm.

cs.GT

On the Merits of LLM-Based Corpus Enrichment

Generative AI (genAI) technologies -- specifically, large language models (LLMs) -- and search have evolving relations. We argue for a novel perspective: using genAI to enrich a document corpus so as to improve query-based retrieval effectiveness. The enrichment is based on modifying existing documents or generating new ones. As an empirical proof of concept, we use LLMs to generate documents relevant to a topic which are more retrievable than existing ones. In addition, we demonstrate the potential merits of using corpus enrichment for retrieval augmented generation (RAG) and answer attribution in question answering.

cs.IR

Fairness under Competition

Algorithmic fairness has emerged as a central issue in ML, and it has become standard practice to adjust ML algorithms so that they will satisfy fairness requirements such as Equal Opportunity. In this paper we consider the effects of adopting such fair classifiers on the overall level of ecosystem fairness. Specifically, we introduce the study of fairness with competing firms, and demonstrate the failure of fair classifiers in yielding fair ecosystems. Our results quantify the loss of fairness in systems, under a variety of conditions, based on classifiers' correlation and the level of their data overlap. We show that even if competing classifiers are individually fair, the ecosystem's outcome may be unfair; and that adjusting biased algorithms to improve their individual fairness may lead to an overall decline in ecosystem fairness. In addition to these theoretical results, we also provide supporting experimental evidence. Together, our model and results provide a novel and essential call for action.

cs.LG

Data Sharing with a Generative AI Competitor

As GenAI platforms grow, their dependence on content from competing providers, combined with access to alternative data sources, creates new challenges for data-sharing decisions. In this paper, we provide a model of data sharing between a content creation firm and a GenAI platform that can also acquire content from third-party experts. The interaction is modeled as a Stackelberg game: the firm first decides how much of its proprietary dataset to share with GenAI, and GenAI subsequently determines how much additional data to acquire from external experts. Their utilities depend on user traffic, monetary transfers, and the cost of acquiring additional data from external experts. We characterize the unique subgame perfect equilibrium of the game and uncover a surprising phenomenon: The firm may be willing to pay GenAI to share the firm's own data, leading to a costly data-sharing equilibrium. We further characterize the set of Pareto improving data prices, and show that such improvements occur only when the firm pays to share data. Finally, we study how the price can be set to optimize different design objectives, such as promoting firm data sharing, expert data acquisition, or a balance of both. Our results shed light on the economic forces shaping data-sharing partnerships in the age of GenAI, and provide guidance for platforms, regulators and policymakers seeking to design effective data exchange mechanisms.

cs.GT

Predicting human decisions with behavioral theories and machine learning

Predicting human decisions under risk and uncertainty remains a fundamental challenge across disciplines. Existing models often struggle even in highly stylized tasks like choice between lotteries. We introduce BEAST Gradient Boosting (BEAST-GB), a hybrid model integrating behavioral theory (BEAST) with machine learning. We first present CPC18, a competition for predicting risky choice, in which BEAST-GB won. Then, using two large datasets, we demonstrate BEAST-GB predicts more accurately than neural networks trained on extensive data and dozens of existing behavioral models. BEAST-GB also generalizes robustly across unseen experimental contexts, surpassing direct empirical generalization, and helps refine and improve the behavioral theory itself. Our analyses highlight the potential of anchoring predictions on behavioral theory even in data-rich settings and even when the theory alone falters. Our results underscore how integrating machine learning with theoretical frameworks, especially those-like BEAST-designed for prediction, can improve our ability to predict and understand human behavior.

cs.AI

Robust-IR @ SIGIR 2025: The First Workshop on Robust Information Retrieval

With the advancement of information retrieval (IR) technologies, robustness is increasingly attracting attention. When deploying technology into practice, we consider not only its average performance under normal conditions but, more importantly, its ability to maintain functionality across a variety of exceptional situations. In recent years, the research on IR robustness covers theory, evaluation, methodology, and application, and all of them show a growing trend. The purpose of this workshop is to systematize the latest results of each research aspect, to foster comprehensive communication within this niche domain while also bridging robust IR research with the broader community, and to promote further future development of robust IR. To avoid the one-sided talk of mini-conferences, this workshop adopts a highly interactive format, including round-table and panel discussion sessions, to encourage active participation and meaningful exchange among attendees.

cs.IR