arXiv subjects

T. D. Stanley

Publications and source records attributed to T. D. Stanley.

3 recordsLinked to original sources

arXiv2026

Why the unrestricted weighted least squares should be routinely reported in medical meta-analyses

The unrestricted weighted least squares (UWLS) meta-analysis estimator of mean effect is an alternative to the conventional random-effects model (RE). It is a weighted least squares regression estimator that can be represented as a multiplicative random-effects model. UWLS has been shown to fit medical research better than RE as measured by AIC/BIC widely across Cochrane Database of Systematic Reviews (CDSR). The independence of UWLS's mean and heterogeneity estimators provide small-sample advantages that RE does not possess. Large small-sample biases and uncertainty in RE's heterogeneity variance estimates explain most of RE's relatively poor fit along with RE's boundary problem, where RE's heterogeneity variance is estimated to be zero. We prove that UWLS almost always has superior fit at RE's boundary with uncommon exceptions. UWLS has also been found to have generally superior statistical properties: bias, MSE, and coverage relative to RE across 1,665 simulation designs compiled from four published studies authored by different teams of researchers. A recent study in this journal replicated UWLS's superior goodness of fit widely across both the CDSR and a new set of simulations. Due to reporting and interpretation errors, this recent study calls for the continued use of RE as the default meta-analysis estimator with limited applications of UWLS. We address this recent study's concerns and show why UWLS should be routinely reported in medical meta-analyses.

stat.ME↗

arXiv2026

Do decisions about outliers and influential effects matter? Evidence from 358 behavioral science meta-analyses

Meta-analysts routinely face estimates that look too large or extreme. Yet, how to handle them is left to the reviewer's judgment. The methods for detecting such estimates are well known. What is missing is an informed assessment of how much alternative handling choices might change a meta-analysis' conclusions. We fill this gap by analyzing the effects of four pre-registered handling treatments across 358 behavioral science meta-analyses with at least ten estimates. Each outlier handling treatment is estimated by two estimators (random effects and unrestricted weighted least squares), and compared to the 'do-nothing' baseline on three outcomes: the pooled effect, statistical significance, and whether the effect reaches the smallest effect size of interest (|d| >= 0.20). Our entire analysis and comparison pipelines were pre-registered. Alternative outlier handling treatments have little effect on the meta-analysis mean as the median absolute change in Cohen's d is at most 0.047 and often much less. Yet, at least one of these four treatments in combination with one of these estimators reverses the statistical significance of 11.5% of meta-analyses and the smallest-effect-of-interest assessment in 15.9%. Winsorizing has the least effect and DFBETAS the most. Categorical changes are found almost entirely among results already close to the decision boundary; strongly significant results essentially never change. These findings give applied meta-analysts, methods specialists, and reviewers a reference point for how much this under-reported choice matters and provide yet another reason for meta-analysts to publicly pre-specify their methods and handling treatments.

econ.EM↗

arXiv2023

Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics

Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta-analyses in environmental sciences and psychology, whereas meta-analyses in medicine are contaminated the least. After adjusting for publication selection bias, the median probability of the presence of an effect decreased from 99.9% to 29.7% in economics, from 98.9% to 55.7% in psychology, from 99.8% to 70.7% in environmental sciences, and from 38.0% to 29.7% in medicine. The median absolute effect sizes (in terms of standardized mean differences) decreased from d = 0.20 to d = 0.07 in economics, from d = 0.37 to d = 0.26 in psychology, from d = 0.62 to d = 0.43 in environmental sciences, and from d = 0.24 to d = 0.13 in medicine.

stat.AP↗