Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2509.14396

Friend or Foe: Delegating to an AI Whose Alignment is Unknown

Drew Fudenberg·Annie Liang
View original source ↗Read source document ↗Export metadata ↓

Abstract

AI systems have the potential to improve decision-making, but decision makers face the risk that the AI may be misaligned with their objectives. We study this problem in the context of a treatment decision, where a designer decides which patient attributes to reveal to an AI before receiving a prediction of the patient's need for treatment. Providing the AI with more information increases the benefits of an aligned AI but also amplifies the harm from a misaligned one. We characterize how the designer should select attributes to balance these competing forces, depending on their beliefs about the AI's reliability. We show that the designer should optimally disclose attributes that identify \emph{rare} segments of the population in which the need for treatment is high, and pool the remaining patients.

Explore related subjects

econ.THcs.GT

Keep this discovery

BibTeXRIS

Drew Fudenberg, Annie Liang. 2025-09-17. Friend or Foe: Delegating to an AI Whose Alignment is Unknown. https://arxiv.org/abs/2509.14396

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Social Preferences and Cooperation: Beliefs, Robustness, and the Limits of Altruism

We study a mechanism of cooperation in the Prisoner's Dilemma (PD). Incorporating social preferences as efficiency concerns into the PD game, we study how altruism translates into cooperation. Under complete information, cooperation requires the opponent's altruism to clear a threshold. We then introduce a subjective extension of Bayesian Nash equilibrium that relaxes the Common Prior Assumption, letting players hold heterogeneous, potentially misspecified beliefs about each other's altruistic type. Cooperation then depends on beliefs about altruism rather than altruism itself, and can be sustained even when opponents are, on average, only weakly altruistic. When fear of exploitation dominates the temptation to defect, beliefs about the opponent's cooperation become strategic complements, so a cooperative and an uncooperative equilibrium can coexist under identical payoffs and an identical, correctly specified prior. Using multiplier preferences, we then study how robust this belief-driven cooperation is to model misspecification. Cooperation is fragile: it survives only above a threshold level of confidence in one's own belief, and can unravel even when the belief itself correctly supports cooperation. As a formal extension, the same robust-control apparatus, applied to a player's action choice, nests Nash equilibrium, Bayesian Nash equilibrium, and logit Quantal Response Equilibrium as limiting cases. Cooperation depends less on how altruistic agents are than on what they believe about each other, and how confident they are that this belief is right.

econ.TH↗
arXiv2026

Utility-Level-Dependent Ambiguity

Experimental evidence suggests that ambiguity-sensitive choice can vary systematically with the circumstances of a decision. This paper isolates one channel within a stable preference relation: ambiguity weighting may depend on the act's certainty-equivalent level. After the standard Anscombe-Aumann calibration of consequence utility, a set of behavioral axioms yields a unique continuous family of normalized monotone capacities $\{\nu_v\}_{v\in(0,1)}$. Each nonendpoint act is evaluated by the Choquet integral associated with the capacity at its own interior certainty-equivalent level, while nonendpoint acts on the same indifference surface share the same capacity. Binary event comparisons identify local event weights at each elicited level and trace their cross-level variation, providing tests of the fixed-capacity restriction. Local uncertainty aversion is equivalent to convexity of $\nu_v$ and yields an implicit multiple-priors representation with certainty-equivalent-indexed local cores. Certainty translation invariance holds if and only if the capacity is fixed across levels, recovering the maintained nondegenerate fixed-capacity Choquet expected utility benchmark; global mixture-betweenness yields implicit additive utility, and imposing both restrictions recovers full-support subjective expected utility. The capacity schedule is a reduced-form ambiguity weighting whose variation may reflect changes in ambiguity perception, ambiguity attitude, or both.

econ.TH↗
arXiv2026

The Attention Cost of Stable Matching

In large markets, scarce attention limits partner evaluation and creates allocation loss, which stability magnifies. In an independent random market with average executable degree $d$, unmatched shares fall at rates $e^{-\sqrt d}$ under stability and $e^{-d}$ under maximum matching on the same graph. Changing consideration can make applications rejected in a provisional active-screen computation relevant again. Exact query-neutral implementation must retain allocation-relevant off-screen authorizations; otherwise, missing authorization must be reacquired. Limited-attention deferred acceptance (LA-DA) preserves valid authorizations and reengages eligible pairs. Conditional on exact next-best information and persistent execution rights, adaptive discovery saves a logarithmic factor in reached proposals relative to independent exposure. In an application to speed dating, bilateral reports let us compare stable and maximum matching on restricted graphs, separating missed opportunities from same-graph stability loss. In Chilean school choice, we document 9,502 applicants accepting higher-ranked or new placements through retained rankings.

econ.TH↗