SearcharxivSearch

arXiv subjects

Laura A. Hatfield

Publications and source records attributed to Laura A. Hatfield.

6 recordsLinked to original sources

A formal approach to variable selection in difference-in-differences

Difference-in-differences (DiD) identification relies mainly on a parallel trends assumption about untreated potential outcomes. Researchers often relax this assumption by assuming conditional parallel trends within units with the same covariate values. However, the process of selecting which covariates to include in this assumption is often \emph{ad hoc}. We propose a formal approach to select the variables that support conditional parallel trends based on graphical criteria. We show that the parallel trends assumption is rarely justified without conditioning on covariates, and that unconditional and conditional parallel trends can conflict with one another. We also demonstrate that a time-invariant covariate with a time-invariant effect on the outcome, which might not ordinarily be considered a confounder in DiD, may be a useful conditioning variable. We clarify that adjustment for a post-treatment covariate depends on what causes that covariate to change. Extending our framework to multiple time periods, we distinguish between treatment type and rollout strategy and examine the problem of treatment-confounder feedback. On the estimation side, we argue that the difficulty of incorporating covariates in DiD, often framed as an estimator problem, is more accurately understood as a misalignment between the adjustment set used by the estimator and the adjustment set required for identification. This misalignment affects several popular estimation procedures, and resolving it requires not a change of estimator, but a change in how covariates enter the estimation procedure. We show how to achieve this alignment for all estimators we evaluate.

stat.ME

Nothing to See Here? A non-inferiority approach to parallel trends

Difference-in-differences is a popular method for observational health policy evaluation. It relies on a causal assumption that in the absence of intervention, treatment groups' outcomes would have evolved in parallel to those of comparison groups. Researchers frequently look for parallel trends in the pre-intervention period to bolster confidence in this assumption. The popular "parallel trends test" evaluates a null hypothesis of parallel trends and, failing to find evidence against the null, concludes that the assumption holds. This tightly controls the probability of falsely concluding that trends are not parallel but may have low power to detect non-parallel trends. When used as a screening step, it can also introduce bias in treatment effect estimates. We propose a non-inferiority/equivalence approach that tightly controls the probability of missing large violations of parallel trends measured on the scale of the treatment effect. Our framework nests several common use cases, including linear trend tests and event studies. We show that our approach may induce no or minimal bias when used as a screening step under commonly-assumed error structures, and absent violations, can offer a higher-power alternative to testing treatment effects in more flexible models. We illustrate our ideas by re-considering a study of the impact of the Affordable Care Act's dependent coverage provision.

stat.ME

Do Methodological Birds of a Feather Flock Together?

Quasi-experimental methods have proliferated over the last two decades, as researchers develop causal inference tools for settings in which randomization is infeasible. Two popular such methods, difference-in-differences (DID) and comparative interrupted time series (CITS), compare observations before and after an intervention in a treated group to an untreated comparison group observed over the same period. Both methods rely on strong, untestable counterfactual assumptions. Despite their similarities, the methodological literature on CITS lacks the mathematical formality of DID. In this paper, we use the potential outcomes framework to formalize two versions of CITS - a general version described by Bloom (2005) and a linear version often used in health services research. We then compare these to two corresponding DID formulations - one with time fixed effects and one with time fixed effects and group trends. We also re-analyze three previously published studies using these methods. We demonstrate that the most general versions of CITS and DID impute the same counterfactuals and estimate the same treatment effects. The only difference between these two designs is the language used to describe them and their popularity in distinct disciplines. We also show that these designs diverge when one constrains them using linearity (CITS) or parallel trends (DID). We recommend defaulting to the more flexible versions and provide advice to practitioners on choosing between the more constrained versions by considering the data-generating mechanism. We also recommend greater attention to specifying the outcome model and counterfactuals in papers, allowing for transparent evaluation of the plausibility of causal assumptions.

stat.ME

Confounding and Regression Adjustment in Difference-in-Differences

Difference-in-differences (diff-in-diff) is a study design that compares outcomes of two groups (treated and comparison) at two time points (pre- and post-treatment) and is widely used in evaluating new policy implementations. For instance, diff-in-diff has been used to estimate the effect that increasing minimum wage has on employment rates and to assess the Affordable Care Act's effect on health outcomes. Although diff-in-diff appears simple, potential pitfalls lurk. In this paper, we discuss one such complication: time-varying confounding. We provide rigorous definitions for confounders in diff-in-diff studies and explore regression strategies to adjust for confounding. In simulations, we show how and when regression adjustment can ameliorate confounding for both time-invariant and time-varying covariates. We compare our regression approach to those models commonly fit in applied literature, which often fail to address the time-varying nature of confounding in diff-in-diff.

stat.AP

Combining Dynamic Predictions from Joint Models for Longitudinal and Time-to-Event Data using Bayesian Model Averaging

The joint modeling of longitudinal and time-to-event data is an active area of statistics research that has received a lot of attention in the recent years. More recently, a new and attractive application of this type of models has been to obtain individualized predictions of survival probabilities and/or of future longitudinal responses. The advantageous feature of these predictions is that they are dynamically updated as extra longitudinal responses are collected for the subjects of interest, providing real time risk assessment using all recorded information. The aim of this paper is two-fold. First, to highlight the importance of modeling the association structure between the longitudinal and event time responses that can greatly influence the derived predictions, and second, to illustrate how we can improve the accuracy of the derived predictions by suitably combining joint models with different association structures. The second goal is achieved using Bayesian model averaging, which, in this setting, has the very intriguing feature that the model weights are not fixed but they are rather subject- and time-dependent, implying that at different follow-up times predictions for the same subject may be based on different models.

stat.AP