arXiv subjects

Shixiang Liu

Publications and source records attributed to Shixiang Liu.

6 recordsLinked to original sources

arXiv2026

Adversarial Contamination Meets Hard Thresholding: An Iterative Algorithm with Signal Adaptivity and Minimax Optimality

Pervasive data contamination -- stemming from measurement errors, outliers, or adversarial corruption -- has motivated the development of robust statistical methods. In this context, we propose a two-stage Adversarial Contamination-resistant Iterative Hard Thresholding (AC-IHT) algorithm for high-dimensional regression with contamination. Our nonconvex algorithm achieves minimax near-optimal (up to logarithmic terms) estimation by iteratively updating the coefficient vector and the contamination vector with different thresholding scales. We further demonstrate that our AC-IHT estimator is signal-adaptive: under proper signal conditions, it adaptively attains a sharper estimation rate and more accurate support recovery. Moreover, it enjoys the strong oracle property, laying a theoretical foundation for asymptotic inference. Numerical experiments confirm its superior finite-sample performance. Finally, we discuss theoretical extensions of the proposed procedure to generalized linear models and to heavy-tailed noise settings.

stat.ML↗

arXiv2026

High-dimensional online learning via asynchronous decomposition: Non-divergent results, dynamic regularization, and beyond

Existing high-dimensional online learning methods often face the challenge that their error bounds, or per-batch sample sizes, diverge as the number of data batches increases. To address this issue, we propose an asynchronous decomposition framework that leverages summary statistics to construct a surrogate score function for current-batch learning. This framework is implemented via a dynamic-regularized iterative hard thresholding algorithm, providing a computationally and memory-efficient solution for sparse online optimization. We provide a unified theoretical analysis that accounts for both the streaming computational error and statistical accuracy, establishing that our estimator maintains non-divergent error bounds and $\ell_0$ sparsity across all batches. Furthermore, the proposed estimator adaptively achieves additional gains as batches accumulate, attaining the oracle accuracy as if the entire historical dataset were accessible and the true support were known. These theoretical properties are further illustrated through an example of the generalized linear model.

stat.ML↗

arXiv2025

Learning Joint Graphical Model with Computational Efficiency, Dynamic Regularization, and Adaptation

Multi-sourced datasets are common in studies of variable interactions, for example, individual-level fMRI integration, cross-domain recommendation, etc, where each source induces a related but distinct dependency structure. Joint learning of multiple graphical models (i.e., multiple precision matrices) has emerged as an important tool in analyzing such data. Unlike separate learning, joint learning can leverage shared structural patterns across graphs to yield more accurate results. In this paper, we present an efficient and adaptive method named MIGHT (\textbf{M}ulti-task \textbf{I}terative \textbf{G}raphical \textbf{H}ard \textbf{T}hresholding) to estimate multiple graphs jointly. We reformulate the joint model into a series of multi-task learning problems through a column-by-column manner, and solve these problems using a dynamic regularized algorithm based on iterative hard thresholding. This framework is inherently parallelizable and therefore efficient in computation. Theoretically, we derive the non-asymptotic error bound for the resulting estimator. Furthermore, the proposed algorithm is adaptive to heterogeneous column-wise signal strengths: for nodes with strong signals, our estimator achieves improved error bounds and selection consistency adaptively, and also exhibits asymptotic normality -- properties rarely explored in existing joint learning methods. The performance of our method is illustrated through numerical simulations and real data analysis on a cancer gene-expression RNA-seq dataset.

stat.ME↗

arXiv2025

Exact recovery in the double sparse model: sufficient and necessary signal conditions

The double sparse linear model, which has both group-wise and element-wise sparsity in regression coefficients, has attracted lots of attention recently. This paper establishes the sufficient and necessary relationship between the exact support recovery and the optimal minimum signal conditions in the double sparse model. Specifically, sharply under the proposed signal conditions, a two-stage double sparse iterative hard thresholding procedure achieves exact support recovery with a suitably chosen threshold parameter. Also, this procedure maintains asymptotic normality aligning with an OLS estimator given true support, hence holding the oracle properties. Conversely, we prove that no method can achieve exact support recovery if these signal conditions are violated. This fills a critical gap in the minimax optimality theory on support recovery of the double sparse model. Finally, numerical experiments are provided to support our theoretical findings.

math.ST↗

arXiv2025

Rethinking Hard Thresholding Pursuit: Full Adaptation and Sharp Estimation

Hard Thresholding Pursuit (HTP) has aroused increasing attention for its robust theoretical guarantees and impressive numerical performance in non-convex optimization. In this paper, we introduce a novel tuning-free procedure, named Full-Adaptive HTP (FAHTP), that simultaneously adapts to both the unknown sparsity and signal strength of the underlying model. We provide an in-depth analysis of the iterative thresholding dynamics of FAHTP, offering refined theoretical insights. In specific, under the beta-min condition $\min_{i \in S^*}|{\boldsymbol{\beta}}^*_i| \ge C\sigma (\log p/n)^{1/2}$, we show that the FAHTP achieves oracle estimation rate $\sigma (s^*/n)^{1/2}$, highlighting its theoretical superiority over convex competitors such as LASSO and SLOPE, and recovers the true support set exactly. More importantly, even without the beta-min condition, our method achieves a tighter error bound than the classical minimax rate with high probability. The comprehensive numerical experiments substantiate our theoretical findings, underscoring the effectiveness and robustness of the proposed FAHTP.

math.ST↗

arXiv2023

A minimax optimal approach to high-dimensional double sparse linear regression

In this paper, we focus our attention on the high-dimensional double sparse linear regression, that is, a combination of element-wise and group-wise sparsity. To address this problem, we propose an IHT-style (iterative hard thresholding) procedure that dynamically updates the threshold at each step. We establish the matching upper and lower bounds for parameter estimation, showing the optimality of our proposal in the minimax sense. More importantly, we introduce a fully adaptive optimal procedure designed to address unknown sparsity and noise levels. Our adaptive procedure demonstrates optimal statistical accuracy with fast convergence. Additionally, we elucidate the significance of the element-wise sparsity level $s_0$ as the trade-off between IHT and group IHT, underscoring the superior performance of our method over both. Leveraging the beta-min condition, we establish that our IHT-style procedure can attain the oracle estimation rate and achieve almost full recovery of the true support set at both the element level and group level. Finally, we demonstrate the superiority of our method by comparing it with several state-of-the-art algorithms on both synthetic and real-world datasets.

math.ST↗