arXiv subjects

Björn Bokelmann

Publications and source records attributed to Björn Bokelmann.

4 recordsLinked to original sources

arXiv2024

Heteroscedasticity-aware stratified sampling to improve uplift modeling

In many business applications, including online marketing and customer churn prevention, randomized controlled trials (RCT's) are conducted to investigate on the effect of specific treatment (coupon offers, advertisement mailings,...). Such RCT's allow for the estimation of average treatment effects as well as the training of (uplift) models for the heterogeneity of treatment effects between individuals. The problem with these RCT's is that they are costly and this cost increases with the number of individuals included into the RCT. For this reason, there is research how to conduct experiments involving a small number of individuals while still obtaining precise treatment effect estimates. We contribute to this literature a heteroskedasticity-aware stratified sampling (HS) scheme, which leverages the fact that different individuals have different noise levels in their outcome and precise treatment effect estimation requires more observations from the "high-noise" individuals than from the "low-noise" individuals. By theory as well as by empirical experiments, we demonstrate that our HS-sampling yields significantly more precise estimates of the ATE, improves uplift models and makes their evaluation more reliable compared to RCT data sampled completely randomly. Due to the relative ease of application and the significant benefits, we expect HS-sampling to be valuable in many real-world applications.

stat.ME↗

arXiv2023

The impact of heteroskedasticity on uplift modeling

There are various applications, where companies need to decide to which individuals they should best allocate treatment. To support such decisions, uplift models are applied to predict treatment effects on an individual level. Based on the predicted treatment effects, individuals can be ranked and treatment allocation can be prioritized according to this ranking. An implicit assumption, which has not been doubted in the previous uplift modeling literature, is that this treatment prioritization approach tends to bring individuals with high treatment effects to the top and individuals with low treatment effects to the bottom of the ranking. In our research, we show that heteroskedastictity in the training data can cause a bias of the uplift model ranking: individuals with the highest treatment effects can get accumulated in large numbers at the bottom of the ranking. We explain theoretically how heteroskedasticity can bias the ranking of uplift models and show this process in a simulation and on real-world data. We argue that this problem of ranking bias due to heteroskedasticity might occur in many real-world applications and requires modification of the treatment prioritization to achieve an efficient treatment allocation.

stat.ML↗

arXiv2023

Predictive modeling for limited distributed targets

Many forecasting applications have a limited distributed target variable, which is zero for most observations and positive for the remaining observations. In the econometrics literature, there is much research about statistical model building for limited distributed target variables. Especially, there are two component model approaches, where one model is build for the probability of the target to be positive and one model for the actual value of the target, given that it is positive. However, the econometric literature focuses on effect estimation and does not provide theory for predictive modeling. Nevertheless, some concepts like the two component model approach and Heckmann's sample selection correction also appear in the predictive modeling literature, without a sound theoretical foundation. In this paper, we theoretically analyze predictive modeling for limited dependent variables and derive best practices. By analyzing various real-world data sets, we also use the derived theoretical results to explain which predictive modeling approach works best on which application.

stat.AP↗

arXiv2022

Improving uplift model evaluation on RCT data

Estimating treatment effects is one of the most challenging and important tasks of data analysts. In many applications, like online marketing and personalized medicine, treatment needs to be allocated to the individuals where it yields a high positive treatment effect. Uplift models help select the right individuals for treatment and maximize the overall treatment effect (uplift). A major challenge in uplift modeling concerns model evaluation. Previous literature suggests methods like the Qini curve and the transformed outcome mean squared error. However, these metrics suffer from variance: their evaluations are strongly affected by random noise in the data, which renders their signals, to a certain degree, arbitrary. We theoretically analyze the variance of uplift evaluation metrics and derive possible methods of variance reduction, which are based on statistical adjustment of the outcome. We derive simple conditions under which the variance reduction methods improve the uplift evaluation metrics and empirically demonstrate their benefits on simulated and real-world data. Our paper provides strong evidence in favor of applying the suggested variance reduction procedures by default when evaluating uplift models on RCT data.

stat.ME↗