arXiv subjects

Argon Chen

Publications and source records attributed to Argon Chen.

3 recordsLinked to original sources

arXiv2026

Bias Correction for Relative Importance Measures via Doubly Stochastic Reallocation

Relative importance (RI) analysis quantifies each predictor's contribution to the explained variance of a linear model. General Dominance (GD), a widely used benchmark, requires evaluating $2^p-1$ sub-models and becomes computationally intensive as the number of predictors $p$ grows. Orthogonalization-Reallocation Measures (ORMs), including Relative Weights (RW) and the Green--Carroll--DeSarbo measure (GCD), provide efficient alternatives by assigning importance to orthogonalized predictors and reallocating it to the original predictors. Each, however, has a structural limitation: RW exhibits a leveling problem that compresses differences among predictor importance values, whereas GCD exhibits an a priori bias that systematically favors certain predictors before a response is observed. We show that this bias is governed by the row-sums of the reallocation matrix. A closed-form analysis under compound symmetry relates the reallocations underlying GCD and RW to a GD-based benchmark, showing that homogeneous multicollinearity alone does not induce an a priori bias and formalizing RW's leveling problem as excess shrinkage relative to the benchmark. We correct GCD's bias by mapping its reallocation matrix to a doubly stochastic matrix using the Method of Alternating Projections (MAP) and the Sinkhorn--Knopp (SK) algorithm, yielding GCD-MAP and GCD-SK. Comprehensive simulations show that GCD-SK removes the structural row-sum bias, substantially improves upon GCD, and often outperforms RW when the first principal component is dominant. We conclude with empirical guidelines for selecting among the measures.

stat.ME↗

arXiv2025

Understanding and Using the Relative Importance Measures Based on Orthogonalization and Reallocation

A class of relative importance measures based on orthogonalization and reallocation, ORMs, has been found to effectively approximate the General Dominance index (GD). In particular, Johnson's Relative Weight (RW) has been deemed the most successful ORM in the literature. Nevertheless, the theoretical foundation of the ORMs remains unclear. To further understand the ORMs, we provide a generalized framework that breaks down the ORM into two functional steps: orthogonalization and reallocation. To assess the impact of each step on the performance of ORMs, we conduct extensive Monte Carlo simulations under various predictors' correlation structures and response variable distributions. Our findings reveal that Johnson's minimal transformation consistently outperforms other common orthogonalization methods. We also summarize the performance of reallocation methods under four scenarios of predictors' correlation structures in terms of the first principal component and the variance inflation factor (VIF). This analysis provides guidelines for selecting appropriate reallocation methods in different scenarios, illustrated with real-world dataset examples. Our research offers a deeper understanding of ORMs and provides valuable insights for practitioners seeking to accurately measure variable importance in various modeling contexts.

stat.ME↗

arXiv2025

Variable Selection Using Relative Importance Rankings

Although conceptually related, variable selection and relative importance (RI) analysis have been treated quite differently in the literature. While RI is typically used for post-hoc model explanation, this paper explores its potential for variable or feature ranking and filter-based selection before model creation. Specifically, we anticipate strong performance from the RI measures because they incorporate both direct and combined effects of predictors, addressing a key limitation of marginal correlation, which ignores dependencies among predictors. We implement and evaluate the RI-based variable ranking and selection methods, including a newly proposed RI measure, CRI.Z, with improved computational efficiency relative to conventional RI measures. Through extensive simulations, we first demonstrate how the RI measures more accurately rank the variables than the marginal correlation, especially when there are suppressed or weak predictors. We then show that predictive models built on these rankings are highly competitive, often outperforming state-of-the-art linear-model methods such as the lasso and relaxed lasso. The proposed RI-based methods are particularly effective in challenging cases involving clusters of highly correlated predictors, a setting known to cause failures in many benchmark methods. The practical utility and efficiency of RI-based methods are further demonstrated through two high-dimensional gene expression datasets. Although lasso methods have dominated the recent literature on variable selection, our study reveals that the RI-based method is a powerful and competitive alternative. We believe these underutilized tools deserve greater attention in statistics and machine learning communities. The code is available at: https://github.com/tien-endotchang/RI-variable-selection.

stat.ML↗