arXiv subjects

Annie Liang

Publications and source records attributed to Annie Liang.

At least 19 recordsLinked to original sources

arXiv2026

Creative Ownership in the Age of AI

Copyright law focuses on whether a new work is "substantially similar" to an existing one, but generative AI can closely imitate style without copying content, a capability now central to ongoing litigation. We argue that existing definitions of infringement are ill-suited to this setting and propose a new criterion: a generative AI output infringes on an existing work if it could not have been generated without that work in its training corpus. To operationalize this definition, we model generative systems as closure operators mapping a corpus of existing works to an output of new works. AI generated outputs are \emph{permissible} if they do not infringe on any existing work according to our criterion. Our results characterize structural properties of permissible generation and reveal a sharp asymptotic dichotomy: when the process of organic creations is light-tailed, dependence on individual works eventually vanishes, so that regulation imposes no limits on AI generation; with heavy-tailed creations, regulation can be persistently constraining.

econ.TH↗

arXiv2026

How Well Do LLMs Predict Human Behavior? A Measure of their Pretrained Knowledge

Large language models (LLMs) are increasingly used to predict human behavior. We propose a measure for evaluating how much knowledge a pretrained LLM brings to such a prediction: its equivalent sample size, defined as the amount of task-specific data needed to match the predictive accuracy of the LLM. We estimate this measure by comparing the prediction error of a fixed LLM in a given domain to that of flexible machine learning models trained on increasing samples of domain-specific data. We further provide a statistical inference procedure by developing a new asymptotic theory for cross-validated prediction error. Finally, we apply this method to the Panel Study of Income Dynamics. We find that LLMs encode considerable predictive information for some economic variables but much less for others, suggesting that their value as substitutes for domain-specific data differs markedly across settings.

econ.EM↗

arXiv2026

Artificial Intelligence Clones

Large language models, trained on personal data, are increasingly able to mimic individual personalities. These ``AI clones'' or ``AI agents'' have the potential to transform how people search for matches in contexts ranging from marriage to employment. This paper presents a theoretical framework to study the tradeoff between the substantially expanded search capacity of AI representations and their imperfect representation of humans. An individual's personality is modeled as a point in $k$-dimensional Euclidean space, and an individual's AI representation is modeled as a noisy approximation of that personality. I compare two search regimes: Under in person search, each person randomly meets some number of individuals and matches to the most compatible among them; under AI-mediated search, individuals match to the person with the most compatible AI representation. I show that a finite number of in-person encounters yields a better expected match than search over infinite AI representations. Moreover, when personality is sufficiently high-dimensional, simply meeting two people in person is more effective than search on an AI platform, regardless of the size of its candidate pool.

econ.TH↗

arXiv2025

Friend or Foe: Delegating to an AI Whose Alignment is Unknown

AI systems have the potential to improve decision-making, but decision makers face the risk that the AI may be misaligned with their objectives. We study this problem in the context of a treatment decision, where a designer decides which patient attributes to reveal to an AI before receiving a prediction of the patient's need for treatment. Providing the AI with more information increases the benefits of an aligned AI but also amplifies the harm from a misaligned one. We characterize how the designer should select attributes to balance these competing forces, depending on their beliefs about the AI's reliability. We show that the designer should optimally disclose attributes that identify \emph{rare} segments of the population in which the need for treatment is high, and pool the remaining patients.

econ.TH↗

arXiv2025

Using Machine Learning to Generate, Clarify, and Improve Economic Models

Machine learning algorithms can now outperform classic economic models in predicting quantities ranging from bargaining outcomes, to choice under uncertainty, to an individual's future jobs and wages. Yet this predictive accuracy comes at a cost: most machine learning algorithms function as black boxes, offering little insight into \emph{why} outcomes occur. This article asks whether machine learning can guide the development of new economic theories. Economic models serve an important purpose beyond prediction -- they uncover the general mechanisms behind observed behaviors. A model that identifies the causal pathways of economic development is more valuable than one that merely predicts which countries will escape poverty, because it enables policymakers to encourage that development in countries where it might not have happened otherwise. Similarly, a model that predicts imperfectly across many domains can be more valuable than one that is highly accurate in a specific domain, since the former allows insights and data obtained from one setting to inform decisions and policy in another. Applying machine learning algorithms off-the-shelf is unlikely to yield such models. But recent work shows that, when reconceived with the aims of an economic modeler in mind, machine learning methods can improve both prediction and understanding. These approaches range from adversarially training algorithms to expose the limits of existing models, to imposing economic theory as a constraint on algorithmic search. Advances in large language models complement these strategies and open new research directions.

econ.TH↗

arXiv2025

The Transfer Performance of Economic Models

Economists often estimate models using data from a particular domain, e.g. estimating risk preferences in a particular subject pool or for a specific class of lotteries. Whether a model's predictions extrapolate well across domains depends on whether the estimated model has captured generalizable structure. We provide a tractable formulation for this "out-of-domain" prediction problem and define the transfer error of a model based on how well it performs on data from a new domain. We derive finite-sample forecast intervals that are guaranteed to cover realized transfer errors with a user-selected probability when domains are iid, and use these intervals to compare the transferability of economic models and black box algorithms for predicting certainty equivalents. We find that in this application, the black box algorithms we consider outperform standard economic models when estimated and tested on data from the same domain, but the economic models generalize across domains better than the black-box algorithms do.

econ.TH↗

arXiv2025

Testing the Fairness-Accuracy Improvability of Algorithms

Many organizations use algorithms that have a disparate impact, i.e., the benefits or harms of the algorithm fall disproportionately on certain social groups. Addressing an algorithm's disparate impact can be challenging, however, because it is often unclear whether it is possible to reduce this impact without sacrificing other objectives of the organization, such as accuracy or profit. Establishing the improvability of algorithms with respect to multiple criteria is of both conceptual and practical interest: in many settings, disparate impact that would otherwise be prohibited under US federal law is permissible if it is necessary to achieve a legitimate business interest. The question is how a policy-maker can formally substantiate, or refute, this "necessity" defense. In this paper, we provide an econometric framework for testing the hypothesis that it is possible to improve on the fairness of an algorithm without compromising on other pre-specified objectives. Our proposed test is simple to implement and can be applied under any exogenous constraint on the algorithm space. We establish the large-sample validity and consistency of our test, and microfound the test's robustness to manipulation based on a game between a policymaker and the analyst. Finally, we apply our approach to evaluate a healthcare algorithm originally considered by Obermeyer et al. (2019), and quantify the extent to which the algorithm's disparate impact can be reduced without compromising the accuracy of its predictions.

econ.EM↗

arXiv2024

The Value of Context: Human versus Black Box Evaluators

Machine learning algorithms are now capable of performing evaluations previously conducted by human experts (e.g., medical diagnoses). How should we conceptualize the difference between evaluation by humans and by algorithms, and when should an individual prefer one over the other? We propose a framework to examine one key distinction between the two forms of evaluation: Machine learning algorithms are standardized, fixing a common set of covariates by which to assess all individuals, while human evaluators customize which covariates are acquired to each individual. Our framework defines and analyzes the advantage of this customization -- the value of context -- in environments with high-dimensional data. We show that unless the agent has precise knowledge about the joint distribution of covariates, the benefit of additional covariates generally outweighs the value of context.

econ.TH↗

arXiv2024

Algorithm Design: A Fairness-Accuracy Frontier

Algorithm designers increasingly optimize not only for accuracy, but also for the fairness of the algorithm across pre-defined groups. We study the tradeoff between fairness and accuracy for any given set of inputs to the algorithm. We propose and characterize a fairness-accuracy frontier, which consists of the optimal points across a broad range of preferences over fairness and accuracy. Our results identify a simple property of the inputs, group-balance, which qualitatively determines the shape of the frontier. We further study an information-design problem where the designer flexibly regulates the inputs (e.g., by coarsening an input or banning its use) but the algorithm is chosen by another agent. Whether it is optimal to ban an input generally depends on the designer's preferences. But when inputs are group-balanced, then excluding group identity is strictly suboptimal for all designers, and when the designer has access to group identity, then it is strictly suboptimal to exclude any informative input.

econ.TH↗

arXiv2024

Algorithmic Fairness and Social Welfare

Algorithms are increasingly used to guide high-stakes decisions about individuals. Consequently, substantial interest has developed around defining and measuring the ``fairness'' of these algorithms. These definitions of fair algorithms share two features: First, they prioritize the role of a pre-defined group identity (e.g., race or gender) by focusing on how the algorithm's impact differs systematically across groups. Second, they are statistical in nature; for example, comparing false positive rates, or assessing whether group identity is independent of the decision (where both are viewed as random variables). These notions are facially distinct from a social welfare approach to fairness, in particular one based on ``veil of ignorance'' thought experiments in which individuals choose how to structure society prior to the realization of their social identity. In this paper, we seek to understand and organize the relationship between these different approaches to fairness. Can the optimization criteria proposed in the algorithmic fairness literature also be motivated as the choices of someone from behind the veil of ignorance? If not, what properties distinguish either approach to fairness?

econ.TH↗

arXiv2024

Challenges in Variable Importance Ranking Under Correlation

Variable importance plays a pivotal role in interpretable machine learning as it helps measure the impact of factors on the output of the prediction model. Model agnostic methods based on the generation of "null" features via permutation (or related approaches) can be applied. Such analysis is often utilized in pharmaceutical applications due to its ability to interpret black-box models, including tree-based ensembles. A major challenge and significant confounder in variable importance estimation however is the presence of between-feature correlation. Recently, several adjustments to marginal permutation utilizing feature knockoffs were proposed to address this issue, such as the variable importance measure known as conditional predictive impact (CPI). Assessment and evaluation of such approaches is the focus of our work. We first present a comprehensive simulation study investigating the impact of feature correlation on the assessment of variable importance. We then theoretically prove the limitation that highly correlated features pose for the CPI through the knockoff construction. While we expect that there is always no correlation between knockoff variables and its corresponding predictor variables, we prove that the correlation increases linearly beyond a certain correlation threshold between the predictor variables. Our findings emphasize the absence of free lunch when dealing with high feature correlation, as well as the necessity of understanding the utility and limitations behind methods in variable importance estimation.

stat.ML↗

arXiv2023

How Flexible is that Functional Form? Quantifying the Restrictiveness of Theories

We propose a restrictiveness measure for economic models based on how well they fit synthetic data from a pre-defined class. This measure, together with a measure for how well the model fits real data, outlines a Pareto frontier, where models that rule out more regularities, yet capture the regularities that are present in real data, are preferred. To illustrate our approach, we evaluate the restrictiveness of popular models in two laboratory settings -- certainty equivalents and initial play -- and in one field setting -- takeup of microfinance in Indian villages. The restrictiveness measure reveals new insights about each of the models, including that some economic models with only a few parameters are very flexible.

econ.TH↗

arXiv2023

Information and Learning in Economic Theory

These lecture notes accompany a one-semester graduate course on information and learning in economic theory. Topics include common knowledge, Bayesian updating, monotone-likelihood ratio properties, affiliation, the Blackwell order, cost of information, learning and merging of beliefs, model uncertainty, model misspecification, and information design.

econ.TH↗

arXiv2022

Data and Incentives

"Big data" gives markets access to previously unmeasured characteristics of individual agents. Policymakers must decide whether and how to regulate the use of this data. We study how new data affects incentives for agents to exert effort in settings such as the labor market, where an agent's quality is initially unknown but is forecast from an observable outcome. We show that measurement of a new covariate has a systematic effect on the average effort exerted by agents, with the direction of the effect determined by whether the covariate is informative about long-run quality or about a shock to short-run outcomes. For a class of covariates satisfying a statistical property we call strong homoskedasticity, this effect is uniform across agents. More generally, new measurements can impact agents unequally, and we show that these distributional effects have a first-order impact on social welfare.

econ.TH↗

arXiv2021

Dynamically Aggregating Diverse Information

An agent has access to multiple information sources, each of which provides information about a different attribute of an unknown state. Information is acquired continuously -- where the agent chooses both which sources to sample from, and also how to allocate attention across them -- until an endogenously chosen time, at which point a decision is taken. We provide an exact characterization of the optimal information acquisition strategy under weak conditions on the agent's prior belief about the different attributes. We then apply this characterization to derive new results regarding: (1) endogenous information acquisition for binary choice, (2) strategic information provision by biased news sources, and (3) the dynamic consequences of attention manipulation.

econ.TH↗

arXiv2020

Games of Incomplete Information Played By Statisticians

Players are statistical learners who learn about payoffs from data. They may interpret the same data differently, but have common knowledge of a class of learning procedures. I propose a metric for the analyst's "confidence" in a strategic prediction, based on the probability that the prediction is consistent with the realized data. The main results characterize the analyst's confidence in a given prediction as the quantity of data grows large, and provide bounds for small datasets. The approach generates new predictions, e.g. that speculative trade is more likely given high-dimensional data, and that coordination is less likely given noisy data.

econ.TH↗

arXiv2019

Measuring the Completeness of Theories

We use machine learning to provide a tractable measure of the amount of predictable variation in the data that a theory captures, which we call its "completeness." We apply this measure to three problems: assigning certain equivalents to lotteries, initial play in games, and human generation of random sequences. We discover considerable variation in the completeness of existing models, which sheds light on whether to focus on developing better models with the same features or instead to look for new features that will improve predictions. We also illustrate how and why completeness varies with the experiments considered, which highlights the role played in choosing which experiments to run.

econ.TH↗

arXiv2018

Overabundant Information and Learning Traps

We develop a model of social learning from overabundant information: Short-lived agents sequentially choose from a large set of (flexibly correlated) information sources for prediction of an unknown state. Signal realizations are public. We demonstrate two starkly different long-run outcomes: (1) efficient information aggregation, where the community eventually learns as fast as possible; (2) "learning traps," where the community gets stuck observing suboptimal sources and learns inefficiently. Our main results identify a simple property of the signal correlation structure that separates these outcomes. In both regimes, we characterize which sources are observed in the long run and how often.

cs.GT↗