SearcharxivSearch

arXiv subjects

Yongli Qin

Publications and source records attributed to Yongli Qin.

4 recordsLinked to original sources

COSTA: Covariance-Optimized Design and Causal Inference under Network-Temporal Interference

Experiments on networks observed over time face network spillovers, temporal carryover, and dependence deliberately introduced by the design. We propose COSTA---Covariance-Optimized Spatiotemporal Treatment Allocation---a joint Bernoulli design for unit--time assignments. Under common treatment marginals and a nonnegative linear network--temporal exposure model, Horvitz--Thompson bias for the sustained all-treated versus all-control contrast is exactly the negative expected weight of an assignment cut. A covariance-level variance envelope yields an MSE bound that can be optimized directly over assignment covariance. To scale this design, we introduce a thresholded-Gaussian Kronecker parameterization that mirrors the network and temporal exposure operators while preserving valid Bernoulli marginals. We next develop inference theory for the joint effects of designed treatment dependence and interference-induced outcome dependence. Canonical correlations between latent blocks generating separated HT contributions supply the coefficients required by graph-$ψ$ central limit and network-HAC theory; a spectral-floor and far-row-mass condition gives a primitive sufficient check. The framework covers sparse, block, Kronecker, locally factored, and other structured covariance sequences satisfying these conditions. Semi-synthetic RetailRocket and MovieLens experiments show substantial default-setting RMSE reductions and well-calibrated model-assisted design-centered intervals across linear, nonlinear, and demand-substitution outcome surfaces.

stat.ME

Ensuring Trustworthy Online A/B Testing: Addressing Five Key Questions on CUPED

A/B testing has become the gold standard for data-driven decision-making in large-scale online experimentation, providing critical guidance for feature launch, pricing optimization, and user experience enhancement. To maximize statistical sensitivity, many technology companies routinely employ Controlled-experiment Using Pre-Experiment Data (CUPED), a technique that achieves substantial variance reduction while preserving the unbiasedness of estimating the average treatment effect. Despite its widespread adoption, several critical methodological and practical nuances of CUPED remain underexplored. This paper systematically addresses five frequently encountered yet overlooked questions regarding the application of CUPED. First, we provide a comparative analysis of various post-CUPED estimators to identify the optimal adjustment specification. Second, we evaluate the validity of regression-based adjustments and delineate robust variance estimation methods tailored for such frameworks. Finally, we extend our investigation to complex but common scenarios, including multi-arm experiments and two-stage sampling designs. Our findings reveal that in these settings, naive reliance on standard variance estimators can lead to severely misleading inferences. By offering rigorous theoretical insights and extensive experimental validation, this work deepens the conceptual understanding of CUPED. Notably, the recommended methodologies have been successfully deployed and integrated into ByteDance's experimentation platform.

stat.AP

Bridging Control Variates and Regression Adjustment in A/B Testing: From Design-Based to Model-Based Frameworks

A B testing serves as the gold standard for large scale, data driven decision making in online businesses. To mitigate metric variability and enhance testing sensitivity, control variates and regression adjustment have emerged as prominent variance reduction techniques, leveraging pre experiment data to improve estimator performance. Over the past decade, these methods have spawned numerous derivatives, yet their theoretical connections and comparative properties remain underexplored. In this paper, we conduct a comprehensive analysis of their statistical properties, establish a formal bridge between the two frameworks in practical implementations, and extend the investigation from design based to model-based frameworks. Through simulation studies and real world experiments at ByteDance, we validate our theoretical insights across both frameworks. Our work aims to provide rigorous guidance for practitioners in online controlled experiments, addressing critical considerations of internal and external validity. The recommended method control variates with group specific coefficient estimates has been fully implemented and deployed on ByteDance's experimental platform.

stat.ME

Combining Experimental and Historical Data for Policy Evaluation

This paper studies policy evaluation with multiple data sources, especially in scenarios that involve one experimental dataset with two arms, complemented by a historical dataset generated under a single control arm. We propose novel data integration methods that linearly integrate base policy value estimators constructed based on the experimental and historical data, with weights optimized to minimize the mean square error (MSE) of the resulting combined estimator. We further apply the pessimistic principle to obtain more robust estimators, and extend these developments to sequential decision making. Theoretically, we establish non-asymptotic error bounds for the MSEs of our proposed estimators, and derive their oracle, efficiency and robustness properties across a broad spectrum of reward shift scenarios. Numerical experiments and real-data-based analyses from a ridesharing company demonstrate the superior performance of the proposed estimators.

stat.ML