Searcharxiv⌕ Search

arXiv subjects

Clément de Chaisemartin

Publications and source records attributed to Clément de Chaisemartin.

At least 19 recordsLinked to original sources

Estimating treatment-effect heterogeneity across sites, in multi-site randomized experiments with few units per site

Multi-site randomized experiments, which are common in economics and medicine, can reveal how treatment effects vary across settings. However, differences between site-specific estimates reflect both treatment-effect heterogeneity and estimation noise. We build on established measurement-error corrections and develop new ones to study this heterogeneity in experiments with many sites and few units per site. In particular, our results can be used to estimate the variance and centered third moment of treatment effects across sites, relate site effects to untreated outcomes, conduct mediation analyses, assess the roles of provider effectiveness and site responsiveness, and study heterogeneity in local average treatment effects (LATEs) under imperfect compliance. We apply our methods to counseling programs delivered by public and private providers in about 200 French employment offices. Average employment gains of about 2 percentage points conceal cross-site standard deviations of intention-to-treat effects of 8 to 9 percentage points. We also find substantial LATE heterogeneity.Effects are negatively associated with untreated employment rates. The two programs also tend to be more effective in the same sites, consistent with a role for heterogeneous site responsiveness.

econ.EM↗

Difference-in-Differences Estimators When No Unit Remains Untreated

We study difference-in-differences designs in which all units are initially untreated and later receive heterogeneous, strictly positive treatment doses. When quasi-untreated units receive doses close to zero, a difference-in-difference estimand using those units as the control group identifies a weighted average treatment effect per unit of treatment, under a parallel-trends assumption. We develop a nonparametric estimator and bias-corrected confidence intervals for this parameter. We also show that the assumptions justifying the usual two-way fixed-effects estimator imply testable linearity restrictions, and propose specification tests that can guide its use.

econ.EM↗

Is Inference Conditional on Not Rejecting a Pre-test Less Reliable than Unconditional Inference?

Researchers often report an estimate and confidence interval only if a specification test does not reject - for example, only if a pre-trend test supports a difference-in-differences design. Does this make the reported confidence interval less reliable? We show that when the identifying assumptions hold, the answer is no: conditional on passing the pre-test, the confidence interval may over-cover but cannot under-cover. When the assumptions fail, pre-testing can reduce coverage, but under an additional condition it still improves coverage relative to not pre-testing. A calibration to 12 differences-in-differences examples suggests modest effects of pre-testing on coverage.

econ.EM↗

Using Pre-Trends for Inference in Difference-in-Differences

Difference-in-differences (DID) are sometimes estimated with many pre-treatment periods. In such settings, the observed pre-treatment outcome evolutions provide direct information about the magnitude of shocks that could also occur after treatment. This paper proposes a simple inference procedure that uses those pre-trends as the reference distribution for the post-treatment DID. The procedure is closely related to existing conformal inference procedures, but its DID-specific predictor leads to a distinct identifying restriction. Existing procedures assume parallel trends, while this paper's procedure does not require parallel trends.

econ.EM↗

Difference-in-Differences Estimators for Treatments Continuously Distributed at Every Period

When studying the effects of taxes, tariffs, or prices using panel data, the treatment is often continuously distributed in every period. We develop difference-in-differences (DID) estimators for such settings. We partition units into switchers, whose treatment changes between consecutive periods, and stayers, whose treatment remains constant. Under a parallel-trends assumption, we show that the slopes of switchers' potential outcomes with respect to the treatment are nonparametrically identified by DID comparisons between switchers and stayers sharing the same baseline treatment level. Conditioning on the baseline treatment is key, as it ensures that the underlying parallel-trends assumption accommodates time-varying treatment effects. We then study two weighted averages of these slopes, discuss their respective advantages, and propose for each a doubly robust, semiparametrically efficient, and $\sqrt{n}$-consistent estimator. Finally, we extend our framework to instrumental variables and illustrate it by estimating the effects of gasoline taxes on prices and fuel consumption.

econ.EM↗

Randomly Assigned First Differences?

We consider a first-difference regression of an outcome evolution $ΔY$ on a treatment evolution $ΔD$. If the treatment effect changes over time, the regression residual is a function of the period-one treatment $D_{1}$. Then, researchers should test if $ΔD$ and $D_{1}$ are correlated: if they are, the regression may suffer from an omitted variable bias. To solve it, researchers may control for $E(ΔD|D_{1})$. We revisit Acemoglu et al (2016), who study the effect of imports from China on US employment. $ΔD$ and $D_{1}$ are correlated. $ΔD$'s coefficient is less negative when controlling for $E(ΔD|D_{1})$.

econ.EM↗

Difference-in-Differences Estimators of Intertemporal Treatment Effects

We study treatment-effect estimation using panel data. The treatment may be non-binary, non-absorbing, and the outcome may be affected by treatment lags. We make a parallel-trends assumption, and propose event-study estimators of the effect of being exposed to a weakly higher treatment dose for $\ell$ periods. We also propose normalized estimators, that estimate a weighted average of the effects of the current treatment and its lags. We also analyze commonly-used two-way fixed-effects regressions. Unlike our estimators, they can be biased in the presence of heterogeneous treatment effects. A local-projection version of those regressions is biased even with homogeneous effects.

econ.EM↗

Treatment-Effect Estimation in Complex Designs under a Parallel-trends Assumption

This paper considers the identification of dynamic treatment effects with panel data, in complex designs where the treatment may not be binary and may not be absorbing. We first show that under no-anticipation and parallel-trends assumptions, we can identify event-study effects comparing outcomes under the actual treatment path and under the status-quo path where all units would have kept their period-one treatment throughout the panel. Those effects can be helpful to evaluate ex-post the policies that effectively took place, and once properly normalized they estimate weighted averages of marginal effects of the current and lagged treatments on the outcome. Yet, they may still be hard to interpret, and they cannot be used to evaluate the effects of other policies than the ones that were conducted. To make progress, we impose another restriction, namely a random coefficients distributed-lag linear model, where effects remain constant over time. Under this model, the usual distributed-lag two-way-fixed-effects regression may be misleading. Instead, we show that this random coefficients model can be estimated simply. We illustrate our findings by revisiting Gentzkow, Shapiro and Sinkinson (2011).

econ.EM↗

Trading-off Bias and Variance When the Size of the Treatment Effect is Bounded

Assume that one is interested in estimating an average treatment effect (ATE), equal to a weighted average of $S$ conditional average treatment effects (CATEs). One has unbiased estimators of the CATEs. One could just average the CATE estimators, to form an unbiased estimator of the ATE. However, some CATE estimators may be less precise than others. Then, downweighting the imprecisely estimated CATEs may lead to a lower mean-squared error and/or shorter confidence intervals. This paper investigates this bias-variance trade-off, by deriving minimax-linear estimators of, and confidence intervals (CI) for, the ATE, under various restrictions on the CATEs. First, I assume that the magnitude of the CATEs is bounded. Then I assume that their heterogeneity is bounded. I use my results to revisit two empirical applications, and find that minimax-linear estimators and CIs lead to small but non-negligible precision gains, of around 5\%.

econ.EM↗

Using did_multiplegt_dyn to Estimate Event-Study Effects in Complex Designs: Overview, and Four Examples Based on Real Datasets

The command did_multiplegt_dyn can be used to estimate event-study effects in complex designs with a potentially non-binary and/or non-absorbing treatment. This paper starts by providing an overview of the estimators computed by the command. Then, simulations based on three real datasets are used to demonstrate the estimators' properties. Finally, the command is used on four real datasets to estimate event-study effects in complex designs. The first example has a binary treatment that can turn on an off. The second example has a continuous absorbing treatment. The third example has a discrete multivalued treatment that can increase or decrease multiple times over time. The fourth example has two, binary and absorbing treatments, where the second treatment always happens after the first.

econ.EM↗

Difference-in-Differences Estimators with Continuous Treatments and no Stayers

Many treatments or policy interventions are continuous in nature. Examples include prices, taxes or temperatures. Empirical researchers have usually relied on two-way fixed effect regressions to estimate treatment effects in such cases. However, such estimators are not robust to heterogeneous treatment effects in general; they also rely on the linearity of treatment effects. We propose estimators for continuous treatments that do not impose those restrictions, and that can be used when there are no stayers: the treatment of all units changes from one period to the next. We start by extending the nonparametric results of de Chaisemartin et al. (2023) to cases without stayers. We also present a parametric estimator, and use it to revisit Deschênes and Greenstone (2012).

econ.EM↗

More Robust Estimators for Instrumental-Variable Panel Designs, With An Application to the Effect of Imports from China on US Employment

We show that first-difference two-stages-least-squares regressions identify non-convex combinations of location-and-period-specific treatment effects. Thus, those regressions could be biased if effects are heterogeneous. We propose an alternative instrumental-variable correlated-random-coefficient (IV-CRC) estimator, that is more robust to heterogeneous effects. We revisit Autor et al. (2013), who use a first-difference two-stages-least-squares regression to estimate the effect of imports from China on US manufacturing employment. Their regression estimates a highly non-convex combination of effects. Our more robust IV-CRC estimator is small and insignificant. Though its confidence interval is wide, it significantly differs from the first-difference two-stages-least-squares estimator.

econ.EM↗

At What Level Should One Cluster Standard Errors in Paired and Small-Strata Experiments?

In matched-pairs experiments in which one cluster per pair of clusters is assigned to treatment, to estimate treatment effects, researchers often regress their outcome on a treatment indicator and pair fixed effects, clustering standard errors at the unit-ofrandomization level. We show that even if the treatment has no effect, a 5%-level t-test based on this regression will wrongly conclude that the treatment has an effect up to 16.5% of the time. To fix this problem, researchers should instead cluster standard errors at the pair level. Using simulations, we show that similar results apply to clustered experiments with small strata.

econ.EM↗

Two-way Fixed Effects and Differences-in-Differences Estimators with Several Treatments

We study two-way-fixed-effects regressions (TWFE) with several treatment variables. Under a parallel trends assumption, we show that the coefficient on each treatment identifies a weighted sum of that treatment's effect, with possibly negative weights, plus a weighted sum of the effects of the other treatments. Thus, those estimators are not robust to heterogeneous effects and may be contaminated by other treatments' effects. We further show that omitting a treatment from the regression can actually reduce the estimator's bias, unlike what would happen under constant treatment effects. We propose an alternative difference-in-differences estimator, robust to heterogeneous effects and immune to the contamination problem. In the application we consider, the TWFE regression identifies a highly non-convex combination of effects, with large contamination weights, and one of its coefficients significantly differs from our heterogeneity-robust estimator.

econ.EM↗

Clustering and External Validity in Randomized Controlled Trials

The randomization inference literature studying randomized controlled trials (RCTs) assumes that units' potential outcomes are deterministic. This assumption is unlikely to hold, as stochastic shocks may take place during the experiment. In this paper, we consider the case of an RCT with individual-level treatment assignment, and we allow for individual-level and cluster-level (e.g. village-level) shocks. We show that one can draw inference on the ATE conditional on the realizations of the cluster-level shocks, using heteroskedasticity-robust standard errors, or on the ATE netted out of those shocks, using cluster-robust standard errors.

econ.EM↗

Two-Way Fixed Effects and Differences-in-Differences with Heterogeneous Treatment Effects: A Survey

Linear regressions with period and group fixed effects are widely used to estimate policies' effects: 26 of the 100 most cited papers published by the American Economic Review from 2015 to 2019 estimate such regressions. It has recently been shown that those regressions may produce misleading estimates, if the policy's effect is heterogeneous between groups or over time, as is often the case. This survey reviews a fast-growing literature that documents this issue, and that proposes alternative estimators robust to heterogeneous effects. We use those alternative estimators to revisit Wolfers (2006).

econ.EM↗

Empirical MSE Minimization to Estimate a Scalar Parameter

We consider the estimation of a scalar parameter, when two estimators are available. The first is always consistent. The second is inconsistent in general, but has a smaller asymptotic variance than the first, and may be consistent if an assumption is satisfied. We propose to use the weighted sum of the two estimators with the lowest estimated mean-squared error (MSE). We show that this third estimator dominates the other two from a minimax-regret perspective: the maximum asymptotic-MSE-gain one may incur by using this estimator rather than one of the other estimators is larger than the maximum asymptotic-MSE-loss.

math.ST↗

BCG vaccination in infancy does not protect against COVID-19. Evidence from a natural experiment in Sweden

The Bacille Calmette-Guérin (BCG) tuberculosis vaccine has immunity benefits against respiratory infections. Accordingly, it has been hypothesized that it may have a protective effect against COVID-19. Recent research found that countries with universal Bacillus Calmette-Guérin (BCG) childhood vaccination policies tend to be less affected by the COVID-19 pandemic. However, such ecological studies are biased by numerous confounders. Instead, this paper takes advantage of a rare nationwide natural experiment that took place in Sweden in 1975, where discontinuation of newborns BCG vaccination led to a dramatic fall of the BCG coverage rate from 92% to 2% , thus allowing us to estimate the BCG's effect without all the biases associated with cross-country comparisons. Numbers of COVID-19 cases and hospitalizations were recorded for birth cohorts born just before and just after that change, representing 1,026,304 and 1,018,544 individuals, respectively. We used regression discontinuity to assess the effect of BCG vaccination on Covid-19 related outcomes. This method used on such a large population allows for a high precision that would be hard to achieve using a classical randomized controlled trial. The odds ratio for Covid-19 cases and Covid-19 related hospitalizations were 0.9997 (CI95: [0.8002-1.1992]) and 1.1931 (CI95: [0.7558-1.6304]), respectively. We can thus reject with 95\% confidence that universal BCG vaccination reduces the number of cases by more than 20% and the number of hospitalizations by more than 24%. While the effect of a recent vaccination must be evaluated, we provide strong evidence that receiving the BCG vaccine at birth does not have a protective effect against COVID-19.

q-bio.PE↗