arXiv subjects

Brantly Callaway

Publications and source records attributed to Brantly Callaway.

18 recordsLinked to original sources

arXiv2026

Difference-in-differences with "bad controls"

This paper considers difference-in-differences identification strategies when the parallel trends assumption holds after conditioning on covariates that may themselves be affected by the treatment (often referred to as "bad controls"). We show that common approaches such as simply dropping bad controls are often ill-advised and develop two alternative approaches that allow bad controls to function as genuine controls despite being affected by treatment. First, we derive explicit conditions that rationalize conditioning only on pre-treatment values of the bad control, leading naturally to the Callaway and Sant'Anna (2021) estimator with pre-treatment values as covariates. Second, under a covariate unconfoundedness condition, we develop imputation and double/debiased machine learning estimators that recover the average treatment effect on the treated. We extend these results to staggered treatment adoption, provide pre-tests for the identifying assumptions, and apply the methods to study the effects of job displacement on earnings.

econ.EM↗

arXiv2025

Beyond Parallel Trends: An Identification-Strategy-Robust Approach to Causal Inference with Panel Data

In this paper, we propose a new approach to causal inference with panel data. Instead of using panel data to adjust for differences in the distribution of unobserved heterogeneity between the treated and comparison groups, we instead use panel data to search for "close comparison groups" -- groups that are similar to the treated group in terms of pre-treatment outcomes. Then, we compare the outcomes of the treated group to the outcomes of these close comparison groups in post-treatment periods. We show that this approach is often identification-strategy-robust in the sense that our approach recovers the ATT under many different non-nested panel data identification strategies, including difference-in-differences, change-in-changes, or lagged outcome unconfoundedness, among several others. We provide related, though non-nested, results under "time homogeneity", where outcomes do not systematically change over time for any comparison group. Our strategy asks more out of the research design -- namely that there exist close comparison groups or time homogeneity (neither of which is required for most existing panel data approaches to causal inference) -- but, when available, leads to more credible inferences.

econ.EM↗

arXiv2025

Causal Inference for Aggregated Treatment

In this paper, we study causal inference when the treatment variable is an aggregation of multiple sub-treatment variables. Researchers often report marginal causal effects for the aggregated treatment, implicitly assuming that the target parameter corresponds to a well-defined average of sub-treatment effects. We show that, even in an ideal scenario for causal inference such as random assignment, the weights underlying this average have some key undesirable properties: they are not unique, they can be negative, and, holding all else constant, these issues become exponentially more likely to occur as the number of sub-treatments increases and the support of each sub-treatment grows. We propose approaches to avoid these problems, depending on whether or not the sub-treatment variables are observed.

econ.EM↗

arXiv2025

Difference-in-Differences Designs: A Practitioner's Guide

Difference-in-differences (DiD) is arguably the most popular quasi-experimental research design. Its canonical form, with two groups and two periods, is well-understood. However, empirical practices can be ad hoc when researchers go beyond that simple case. This article provides an organizing framework for discussing different types of DiD designs and their associated DiD estimators. It discusses covariates, weights, handling multiple periods, and staggered treatments. The organizational framework, however, applies to other extensions of DiD methods as well.

econ.EM↗

arXiv2024

Testing the effects of an unobservable factor: Do marriage prospects affect college major choice?

This paper develops an econometric test for whether an unobservable factor jointly influences a choice among multiple alternatives and a subsequent binary outcome. Because only transformed versions of the choice model's unobservables are identified, we combine a copula-based joint model with results on the covariance between ordered and unordered normal variables to recover the covariance between structural shocks, yielding a simulation-based test for a common unobserved factor. Applying it to the National Longitudinal Survey of Youth 1997, we find unobserved marriage prospects significantly influence college major choice. Supporting this, the dependence disappears once stated marriage expectations, elicited before major selection, are included.

econ.EM↗

arXiv2024

Difference-in-Differences when Parallel Trends Holds Conditional on Covariates

We consider difference-in-differences identification and estimation strategies when the parallel trends assumption holds conditional on covariates, which can be time-varying, time-invariant, or both. We uncover several weaknesses of two-way fixed effects (TWFE) regressions in this context. The most important, which we call \textit{hidden linearity bias}, arises because transformations that eliminate unit fixed effects also transform the covariates, either implicitly changing the identification strategy or relying on correct model specification. We provide diagnostics for assessing a TWFE regression's susceptibility to hidden linearity bias and propose alternative estimation strategies that circumvent these issues.

econ.EM↗

arXiv2023

Treatment Effects in Staggered Adoption Designs with Non-Parallel Trends

This paper considers identifying and estimating causal effect parameters in a staggered treatment adoption setting -- that is, where a researcher has access to panel data and treatment timing varies across units. We consider the case where untreated potential outcomes may follow non-parallel trends over time across groups. This implies that the identifying assumptions of leading approaches such as difference-in-differences do not hold. We mainly focus on the case where untreated potential outcomes are generated by an interactive fixed effects model and show that variation in treatment timing provides additional moment conditions that can be used to recover a large class of target causal effect parameters. Our approach exploits the variation in treatment timing without requiring either (i) a large number of time periods or (ii) requiring any extra exclusion restrictions. This is in contrast to essentially all of the literature on interactive fixed effects models which requires at least one of these extra conditions. Rather, our approach directly applies in settings where there is variation in treatment timing. Although our main focus is on a model with interactive fixed effects, our idea of using variation in treatment timing to recover causal effect parameters is quite general and could be adapted to other settings with non-parallel trends across groups such as dynamic panel data models.

econ.EM↗

arXiv2022

Difference-in-Differences for Policy Evaluation

Difference-in-differences is one of the most used identification strategies in empirical work in economics. This chapter reviews a number of important, recent developments related to difference-in-differences. First, this chapter reviews recent work pointing out limitations of two way fixed effects regressions (these are panel data regressions that have been the dominant approach to implementing difference-in-differences identification strategies) that arise in empirically relevant settings where there are more than two time periods, variation in treatment timing across units, and treatment effect heterogeneity. Second, this chapter reviews recently proposed alternative approaches that are able to circumvent these issues without being substantially more complicated to implement. Third, this chapter covers a number of extensions to these results, paying particular attention to (i) parallel trends assumptions that hold only after conditioning on observed covariates and (ii) strategies to partially identify causal effect parameters in difference-in-differences applications in cases where the parallel trends assumption may be violated.

econ.EM↗

arXiv2022

Treatment Effects in Interactive Fixed Effects Models with a Small Number of Time Periods

This paper considers identifying and estimating the Average Treatment Effect on the Treated (ATT) when untreated potential outcomes are generated by an interactive fixed effects model. That is, in addition to time-period and individual fixed effects, we consider the case where there is an unobserved time invariant variable whose effect on untreated potential outcomes may change over time and which can therefore cause outcomes (in the absence of participating in the treatment) to follow different paths for the treated group relative to the untreated group. The models that we consider in this paper generalize many commonly used models in the treatment effects literature including difference in differences and individual-specific linear trend models. Unlike the majority of the literature on interactive fixed effects models, we do not require the number of time periods to go to infinity to consistently estimate the ATT. Our main identification result relies on having the effect of some time invariant covariate (e.g., race or sex) not vary over time. Using our approach, we show that the ATT can be identified with as few as three time periods and with panel or repeated cross sections data.

econ.EM↗

arXiv2022

Difference in Differences with Time-Varying Covariates

This paper considers identification and estimation of causal effect parameters from participating in a binary treatment in a difference in differences (DID) setup when the parallel trends assumption holds after conditioning on observed covariates. Relative to existing work in the econometrics literature, we consider the case where the value of covariates can change over time and, potentially, where participating in the treatment can affect the covariates themselves. We propose new empirical strategies in both cases. We also consider two-way fixed effects (TWFE) regressions that include time-varying regressors, which is the most common way that DID identification strategies are implemented under conditional parallel trends. We show that, even in the case with only two time periods, these TWFE regressions are not generally robust to (i) time-varying covariates being affected by the treatment, (ii) treatment effects and/or paths of untreated potential outcomes depending on the level of time-varying covariates in addition to only the change in the covariates over time, (iii) treatment effects and/or paths of untreated potential outcomes depending on time-invariant covariates, (iv) treatment effect heterogeneity with respect to observed covariates, and (v) violations of strong functional form assumptions, both for outcomes over time and the propensity score, that are unlikely to be plausible in most DID applications. Thus, TWFE regressions can deliver misleading estimates of causal effect parameters in a number of empirically relevant cases. We propose both doubly robust estimands and regression adjustment/imputation strategies that are robust to these issues while not being substantially more challenging to implement.

econ.EM↗

arXiv2021

Distributional Effects with Two-Sided Measurement Error: An Application to Intergenerational Income Mobility

This paper considers identification and estimation of distributional effect parameters that depend on the joint distribution of an outcome and another variable of interest ("treatment") in a setting with "two-sided" measurement error -- that is, where both variables are possibly measured with error. Examples of these parameters in the context of intergenerational income mobility include transition matrices, rank-rank correlations, and the poverty rate of children as a function of their parents' income, among others. Building on recent work on quantile regression (QR) with measurement error in the outcome (particularly, Hausman, Liu, Luo, and Palmer (2021)), we show that, given (i) two linear QR models separately for the outcome and treatment conditional on other observed covariates and (ii) assumptions about the measurement error for each variable, one can recover the joint distribution of the outcome and the treatment. Besides these conditions, our approach does not require an instrument, repeated measurements, or distributional assumptions about the measurement error. Using recent data from the 1997 National Longitudinal Study of Youth, we find that accounting for measurement error notably reduces several estimates of intergenerational mobility parameters.

econ.EM↗

arXiv2021

Difference-in-Differences with a Continuous Treatment

This paper analyzes difference-in-differences designs with a continuous treatment. We show that treatment-on-the-treated-type parameters are identified under a parallel trends assumption analogous to the binary treatment case. However, comparing these parameters across treatments is challenging because parallel trends does not rule out selection bias. We discuss alternative, typically stronger, assumptions that eliminate selection bias. We further show that popular two-way fixed effects estimands admit multiple interpretations, depending on the underlying causal building block, all having important limitations as meaningful summaries of treatment effects. Finally, we introduce alternative estimation procedures that avoid these drawbacks and demonstrate them in an empirical application.

econ.EM↗

arXiv2021

Policy Evaluation during a Pandemic

National and local governments have implemented a large number of policies in response to the Covid-19 pandemic. Evaluating the effects of these policies, both on the number of Covid-19 cases as well as on other economic outcomes is a key ingredient for policymakers to be able to determine which policies are most effective as well as the relative costs and benefits of particular policies. In this paper, we consider the relative merits of common identification strategies that exploit variation in the timing of policies across different locations by checking whether the identification strategies are compatible with leading epidemic models in the epidemiology literature. We argue that unconfoundedness type approaches, that condition on the pre-treatment "state" of the pandemic, are likely to be more useful for evaluating policies than difference-in-differences type approaches due to the highly nonlinear spread of cases during a pandemic. For difference-in-differences, we further show that a version of this problem continues to exist even when one is interested in understanding the effect of a policy on other economic outcomes when those outcomes also depend on the number of Covid-19 cases. We propose alternative approaches that are able to circumvent these issues. We apply our proposed approach to study the effect of state level shelter-in-place orders early in the pandemic.

econ.EM↗

arXiv2020

Difference-in-Differences with Multiple Time Periods

In this article, we consider identification, estimation, and inference procedures for treatment effect parameters using Difference-in-Differences (DiD) with (i) multiple time periods, (ii) variation in treatment timing, and (iii) when the "parallel trends assumption" holds potentially only after conditioning on observed covariates. We show that a family of causal effect parameters are identified in staggered DiD setups, even if differences in observed characteristics create non-parallel outcome dynamics between groups. Our identification results allow one to use outcome regression, inverse probability weighting, or doubly-robust estimands. We also propose different aggregation schemes that can be used to highlight treatment effect heterogeneity across different dimensions as well as to summarize the overall effect of participating in the treatment. We establish the asymptotic properties of the proposed estimators and prove the validity of a computationally convenient bootstrap procedure to conduct asymptotically valid simultaneous (instead of pointwise) inference. Finally, we illustrate the relevance of our proposed tools by analyzing the effect of the minimum wage on teen employment from 2001--2007. Open-source software is available for implementing the proposed methods.

econ.EM↗

arXiv2020

Heterogeneous Effects of Job Displacement on Earnings

This paper considers how the effect of job displacement varies across different individuals. In particular, our interest centers on features of the distribution of the individual-level effect of job displacement. Identifying features of this distribution is particularly challenging -- e.g., even if we could randomly assign workers to be displaced or not, many of the parameters that we consider would not be point identified. We exploit our access to panel data, and our approach relies on comparing outcomes of displaced workers to outcomes the same workers would have experienced if they had not been displaced and if they maintained the same rank in the distribution of earnings as they had before they were displaced. Using data from the Displaced Workers Survey, we find that displaced workers earn about $157 per week less, on average, than they would have earned if they had not been displaced. We also find that there is substantial heterogeneity. We estimate that 42% of workers have higher earnings than they would have had if they had not been displaced and that a large fraction of workers have experienced substantially more negative effects than the average effect of displacement. Finally, we also document major differences in the distribution of the effect of job displacement across education levels, sex, age, and counterfactual earnings levels. Throughout the paper, we rely heavily on quantile regression. First, we use quantile regression as a flexible (yet feasible) first step estimator of conditional distributions and quantile functions that our main results build on. We also use quantile regression to study how covariates affect the distribution of the individual-level effect of job displacement.

econ.EM↗

arXiv2020

Bounds on Distributional Treatment Effect Parameters using Panel Data with an Application on Job Displacement

This paper develops new techniques to bound distributional treatment effect parameters that depend on the joint distribution of potential outcomes -- an object not identified by standard identifying assumptions such as selection on observables or even when treatment is randomly assigned. I show that panel data and an additional assumption on the dependence between untreated potential outcomes for the treated group over time (i) provide more identifying power for distributional treatment effect parameters than existing bounds and (ii) provide a more plausible set of conditions than existing methods that obtain point identification. I apply these bounds to study heterogeneity in the effect of job displacement during the Great Recession. Using standard techniques, I find that workers who were displaced during the Great Recession lost on average 34\% of their earnings relative to their counterfactual earnings had they not been displaced. Using the methods developed in the current paper, I also show that the average effect masks substantial heterogeneity across workers.

econ.EM↗

arXiv2020

Evaluating Policies Early in a Pandemic: Bounding Policy Effects with Nonrandomly Missing Data

During the early part of the Covid-19 pandemic, national and local governments introduced a number of policies to combat the spread of Covid-19. In this paper, we propose a new approach to bound the effects of such early-pandemic policies on Covid-19 cases and other outcomes while dealing with complications arising from (i) limited availability of Covid-19 tests, (ii) differential availability of Covid-19 tests across locations, and (iii) eligibility requirements for individuals to be tested. We use our approach study the effects of Tennessee's expansion of Covid-19 testing early in the pandemic and find that the policy decreased Covid-19 cases.

econ.EM↗

arXiv2017

Quantile Treatment Effects in Difference in Differences Models under Dependence Restrictions and with only Two Time Periods

This paper shows that the Conditional Quantile Treatment Effect on the Treated can be identified using a combination of (i) a conditional Distributional Difference in Differences assumption and (ii) an assumption on the conditional dependence between the change in untreated potential outcomes and the initial level of untreated potential outcomes for the treated group. The second assumption recovers the unknown dependence from the observed dependence for the untreated group. We also consider estimation and inference in the case where all of the covariates are discrete. We propose a uniform inference procedure based on the exchangeable bootstrap and show its validity. We conclude the paper by estimating the effect of state-level changes in the minimum wage on the distribution of earnings for subgroups defined by race, gender, and education.

stat.ME↗