Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2306.17369

Adaptive sieving: A dimension reduction technique for sparse optimization problems

Yancheng Yuan·Meixia Lin·Defeng Sun·Kim-Chuan Toh
View original source ↗Read source document ↗Export metadata ↓

Abstract

In this paper, we propose an adaptive sieving (AS) strategy for solving general sparse machine learning models by effectively exploring the intrinsic sparsity of the solutions, wherein only a sequence of reduced problems with much smaller sizes need to be solved. We further apply the proposed AS strategy to generate solution paths for large-scale sparse optimization problems efficiently. We establish the theoretical guarantees for the proposed AS strategy including its finite termination property. Extensive numerical experiments are presented in this paper to demonstrate the effectiveness and flexibility of the AS strategy to solve large-scale machine learning models.

Explore related subjects

math.OC

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Yancheng Yuan, Meixia Lin, Defeng Sun, Kim-Chuan Toh. 2025-04-25. Adaptive sieving: A dimension reduction technique for sparse optimization problems. https://arxiv.org/abs/2306.17369

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Unified Convergence Theory of Stochastic and Variance-Reduced Cubic Newton Methods

We study stochastic Cubic Newton methods for solving general possibly non-convex minimization problems. We propose a new framework, which we call the helper framework, that provides a unified view of the stochastic and variance-reduced second-order algorithms equipped with global complexity guarantees. It can also be applied to learning with auxiliary information. Our helper framework offers the algorithm designer high flexibility for constructing and analyzing the stochastic Cubic Newton methods, allowing arbitrary size batches, and the use of noisy and possibly biased estimates of the gradients and Hessians, incorporating both the variance reduction and the lazy Hessian updates. We recover the best-known complexities for the stochastic and variance-reduced Cubic Newton, under weak assumptions on the noise. A direct consequence of our theory is the new lazy stochastic second-order method, which significantly improves the arithmetic complexity for large dimension problems. We also establish complexity bounds for the classes of gradient-dominated objectives, that include convex and strongly convex problems. For Auxiliary Learning, we show that using a helper (auxiliary function) can outperform training alone if a given similarity measure is small.

math.OC↗
arXiv2026

The Riemannian Convex Bundle Method

We introduce the convex bundle method to solve convex, non-smooth optimization problems on Riemannian manifolds of bounded sectional curvature. Each step of our method is based on a model that involves the convex hull of previously collected subgradients, parallelly transported into the current serious iterate. This approach generalizes the dual form of classical bundle subproblems in Euclidean space. We prove that, under mild conditions, the convex bundle method converges to a minimizer. Several numerical examples implemented using Manopt$.$jl illustrate the performance of the proposed method and compare it to the subgradient method, the cyclic proximal point algorithm, as well as the proximal bundle method.

math.OC↗
arXiv2026

Leveraging Christoffel-Darboux Kernels to Strengthen Moment-SOS Relaxations

The classical Moment-SOS hierarchy allows one to approximate the global minimum of a polynomial optimization problem through semidefinite relaxations of increasing accuracy. For many optimization problems, however, moving to higher relaxation orders quickly becomes computationally prohibitive. While this difficulty is commonly addressed by exploiting structural properties such as symmetry or sparsity, we explore a complementary direction: strengthening a low-order relaxation using information contained in its own pseudo-moment solution. To this end, we leverage Christoffel-Darboux kernels to identify regions that are compatible with the current pseudo-moment information and use them to construct additional polynomial constraints. We propose two mechanisms: the first, denoted $\HH$, is based on successive global Christoffel restrictions, while the second, denoted $\hh$, combines marginal Christoffel information with an available local solution. For $\HH$, we establish a uniform separation property for successive pseudo-moment solutions and geometric convergence of the fixed-order bounds on a particular subclass of knapsack problems. Moreover, for specific Boolean quadratic problems, we derive an explicit $\hh$ selection rule and establish conditions under which the resulting marginal constraints preserve all global minimizers. Numerical experiments on dense and sparse nonconvex quadratically constrained quadratic programs show that the resulting strengthened low-order relaxations can significantly improve the initial bounds, sometimes attaining exactness, while remaining substantially cheaper than solving higher-order relaxations.

math.OC↗