arXiv subjects

Philip Marx

Publications and source records attributed to Philip Marx.

6 recordsLinked to original sources

arXiv2025

Misaligned by Design: Incentive Failures in Machine Learning

The cost of error in many high-stakes settings is asymmetric: misdiagnosing pneumonia when absent is an inconvenience, but failing to detect it when present can be life-threatening. Because of this, artificial intelligence (AI) models used to assist such decisions are frequently trained with asymmetric loss functions that incorporate human decision-makers' trade-offs between false positives and false negatives. In two focal applications, we show that this standard alignment practice can backfire. In both cases, it would be better to train the machine learning model with a loss function that ignores the human's objective and then adjust predictions ex post according to that objective. We rationalize this result using an economic model of incentive design with endogenous information acquisition. The key insight from our theoretical framework is that machine classifiers perform not one but two incentivized tasks: choosing how to classify and learning how to classify. We show that while the adjustments engineers use correctly incentivize choosing, they can simultaneously reduce the incentives to learn. Our formal treatment of the problem reveals that methods embraced for their intuitive appeal can in fact misalign human and machine objectives in predictable ways.

econ.GN↗

arXiv2025

Testing Capacity-Constrained Learning

We introduce the first general test of capacity-constrained learning models. Cognitive economic models of this type share the common feature that constraints on perception are exogenously fixed, as in the widely used fixed-capacity versions of rational inattention (Sims 2003) and efficient coding (Woodford 2012). We show that choice data are consistent with capacity-constrained learning if and only if they satisfy a No Improving (Action or Attention) Switches (NIS) condition. Based on existing experiments in which the incentives for being correct are varied, we find strong evidence that participants fail NIS for a wide range of standard perceptual tasks: identifying the proportion of ball colors, recognizing shapes, and counting the number of balls. However, we find that this is not true for all existing perceptual tasks in the literature, which offers insights into settings where we do or do not expect incentives to impact the extent of attention.

econ.GN↗

arXiv2024

Heterogeneous Treatment Effects via Linear Dynamic Panel Data Models

We study the identification of heterogeneous, intertemporal treatment effects (TE) when potential outcomes depend on past treatments. First, applying a dynamic panel data model to observed outcomes, we show that an instrumental variable (IV) version of the estimand in Arellano and Bond (1991) recovers a non-convex (negatively weighted) aggregate of TE plus non-vanishing trends. We then provide conditions on sequential exchangeability (SE) of treatment and on TE heterogeneity that reduce such an IV estimand to a convex (positively weighted) aggregate of TE. Second, even when SE is generically violated, such estimands identify causal parameters when potential outcomes are generated by dynamic panel data models with some homogeneity or mild selection assumptions. Finally, we motivate SE and compare it with parallel trends (PT) in various settings with experimental data (when treatments are sequentially randomized) and observational data (when treatments are dynamic, rational choices under learning).

econ.EM↗

arXiv2022

Parallel Trends and Dynamic Choices

Difference-in-differences is a common method for estimating treatment effects, and the parallel trends condition is its main identifying assumption: the trend in mean untreated outcomes is independent of the observed treatment status. In observational settings, treatment is often a dynamic choice made or influenced by rational actors, such as policy-makers, firms, or individual agents. This paper relates parallel trends to economic models of dynamic choice. We clarify the implications of parallel trends on agent behavior and study when dynamic selection motives lead to violations of parallel trends. Finally, we consider identification under alternative assumptions that accommodate features of dynamic choice.

econ.EM↗

arXiv2022

Calibrating for Class Weights by Modeling Machine Learning

A much studied issue is the extent to which the confidence scores provided by machine learning algorithms are calibrated to ground truth probabilities. Our starting point is that calibration is seemingly incompatible with class weighting, a technique often employed when one class is less common (class imbalance) or with the hope of achieving some external objective (cost-sensitive learning). We provide a model-based explanation for this incompatibility and use our anthropomorphic model to generate a simple method of recovering likelihoods from an algorithm that is miscalibrated due to class weighting. We validate this approach in the binary pneumonia detection task of Rajpurkar, Irvin, Zhu, et al. (2017).

cs.LG↗

arXiv2020

Sharp Bounds in the Latent Index Selection Model

A fundamental question underlying the literature on partial identification is: what can we learn about parameters that are relevant for policy but not necessarily point-identified by the exogenous variation we observe? This paper provides an answer in terms of sharp, analytic characterizations and bounds for an important class of policy-relevant treatment effects, consisting of marginal treatment effects and linear functionals thereof, in the latent index selection model as formalized in Vytlacil (2002). The sharp bounds use the full content of identified marginal distributions, and analytic derivations rely on the theory of stochastic orders. The proposed methods also make it possible to sharply incorporate new auxiliary assumptions on distributions into the latent index selection framework. Empirically, I apply the methods to study the effects of Medicaid on emergency room utilization in the Oregon Health Insurance Experiment, showing that the predictions from extrapolations based on a distribution assumption (rank similarity) differ substantively and consistently from existing extrapolations based on a parametric mean assumption (linearity). This underscores the value of utilizing the model's full empirical content in combination with auxiliary assumptions.

econ.EM↗