arXiv subjects

Anthony Christidis

Publications and source records attributed to Anthony Christidis.

4 recordsLinked to original sources

arXiv2026

Fast Robust Regression via Orthogonal Block Updates

Robust regression methods, particularly MM-estimators, are essential for analyzing datasets where heavy-tailed noise or high-leverage outliers may be present. Algorithms to compute these estimators are iterative and rely on having a good initial point. A widely-used probabilistic approach to obtaining an initial regression estimator that is not affected by outliers consists of fitting linear regression models to many random subsets of the training data. Unfortunately, the number of subsets that need to be considered in order to have a high probability of finding one that is clean of outliers grows exponentially with the number of predictors. This renders the approach unfeasible for models with a moderate to large numbers of variables. Alternative non-stochastic strategies that have been proposed recently also fail to scale well when the number of parameters and observations are large. To overcome this problem, we propose a highly scalable algorithm based on block-coordinate descent. Our Robust Orthogonal Block Updates (ROBU) algorithm uses lower-dimensional blocks of explanatory variables, which require much fewer sub-samples to find a good initial point with high probability. Extensive simulations and a proteogenomics data application illustrate the robustness properties of the estimators computed with ROBU and demonstrate that they compare favourably to those calculated with existing algorithms.

stat.ME↗

arXiv2026

Fast and Scalable Cellwise-Robust Ensembles for High-Dimensional Data

Variable selection and ensemble methods are central to high-dimensional modelling, enabling the identification of relevant predictors and the construction of stable predictive signals through aggregation across multiple models. However, in practice, high-dimensional data are often affected by cellwise contamination, in which individual cells of the data matrix deviate from the underlying multivariate structure without necessarily making the corresponding observation outlying. This type of contamination can easily propagate throughout many observations, compromising variable selection procedures and ensemble methods, including robust methods designed for contamination affecting entire observations (casewise contamination). To address this limitation, we propose the Fast and Scalable Cellwise-Robust Ensemble (FSCRE) algorithm. FSCRE dynamically partitions predictors into disjoint sub-models using a competitive proposer-arbiter architecture operating in a robust correlation framework. Through extensive simulations and a bioinformatics application, we demonstrate FSCRE's competitive performance in variable selection precision, recall, and predictive accuracy across various contamination scenarios, all while maintaining high computational efficiency in high-dimensional settings. This work provides a unified framework connecting cellwise-robust estimation with high-performance ensemble learning, with an implementation available on CRAN.

stat.ME↗

arXiv2019

Split Regularized Regression

We propose an approach for fitting linear regression models that splits the set of covariates into groups. The optimal split of the variables into groups and the regularized estimation of the regression coefficients are performed by minimizing an objective function that encourages sparsity within each group and diversity among them. The estimated coefficients are then pooled together to form the final fit. Our procedure works on top of a given penalized linear regression estimator (e.g., Lasso, elastic net) by fitting it to possibly overlapping groups of features, encouraging diversity among these groups to reduce the correlation of the corresponding predictions. For the case of two groups, elastic net penalty and orthogonal predictors, we give a closed form solution for the regression coefficients in each group. We establish the consistency of our method with the number of predictors possibly increasing with the sample size. An extensive simulation study and real-data applications show that in general the proposed method improves the prediction accuracy of the base estimator used in the procedure. Possible extensions to GLMs and other models are discussed. The supplemental material for this article, available online, contains the proofs of our theoretical results and the full results of our simulation study.

stat.ME↗

arXiv2018

Objective-Driven Ensembles: Bridging the Gap Between Interpretable Sparsity and Algorithmic Prediction

Sparse methods (e.g., Best Subset Selection, Elastic Net) are the standard approach for obtaining interpretable models, but they can suffer from high variance and vulnerability to spurious correlations. Alternatively, algorithmic ensembles (e.g., Random Forests, Gradient Boosting) achieve high prediction accuracy but yield uninterpretable black boxes driven by randomization or sequential residual fitting. In recent years, a unifying paradigm has emerged: Objective-Driven Ensembles. By generalizing best subset selection into a joint mathematical optimization problem, this approach generates interpretable ensembles by optimally splitting predictors across a small number of diverse models. In this paper, we synthesize this growing body of literature and illustrate the statistical principles driving its empirical success. Specifically, we utilize finite-sample bounds to demonstrate how penalizing predictor overlap controls ensemble covariance and provides a mathematical hedge against spurious correlations. We evaluate these mechanics using an exact combinatorial oracle, and review how recent computational approximations have successfully scaled this framework to a variety of domains, including high-dimensional data, classification tasks, and settings with casewise or cellwise contamination, achieving machine-learning-level accuracy while retaining the interpretability of sparse models.

stat.ME↗