Cluster-Level Experiments using Temporal Switchback Designs: Precision Gains in Pricing A/B Tests at LATAM Airlines
Experimentation is central to modern digital businesses, but many operational decisions cannot be randomized at the user level. In such cases, cluster-level experiments, where clusters are usually geographic, come to the rescue. However, such experiments often suffer from low power due to persistent cluster heterogeneity, strong seasonality, and autocorrelated outcome metrics, as well as common shocks that move many clusters simultaneously. On an example of airline pricing - where policies are typically applied at the route level and thus the A/B test unit of analysis is a route - we study switchback designs to remedy these problems. In switchback designs, each cluster (route in our case) alternates between treatment and control on a fixed schedule, creating within-route contrasts that mitigate time-invariant heterogeneity and reduce sensitivity to low-frequency noise. We provide a unified Two-Way Fixed Effects interpretation of switchback experiments that makes the identifying variation explicit after partialling out route and time effects, clarifying how switching cadence interacts with temporal dependence to determine precision. Empirically, we evaluate weekly and daily switchback cadences using calibrated synthetic regimes and operational airline data from ancillary pricing. In our evaluations, switchbacks decrease standard errors by up to 67%, with daily switching yielding the largest gains over short horizons and weekly switching offering a strong and simpler-to-operationalize alternative.