arXiv subjects

Yun-Bin Zhao

Publications and source records attributed to Yun-Bin Zhao.

At least 19 recordsLinked to original sources

arXiv2026

Newton-Direction-Based ReLU-Thresholding Methods for Nonnegative Sparse Signal Recovery

Nonnegative sparse signal recovery has been extensively studied due to its broad applications. Recent work has integrated rectified linear unit (ReLU) techniques to enhance existing recovery algorithms. We merge Newton-type thresholding with ReLU-based approaches to propose two algorithms: Newton-Direction-Based ReLU-Thresholding (NDRT) and its enhanced variant, Newton-Direction-Based ReLU-Thresholding Pursuit (NDRTP). Theoretical analysis iindicates that both algorithms can guarantee exact recovery of nonnegative sparse signals when the measurement matrix satisfies a certain condition.. Numerical experiments demonstrate NDRTP achieves competitive performance compared to several existing methods in both noisy and noiseless scenarios.

eess.SP↗

arXiv2026

Optimality Conditions for Sparse Bilinear Least Squares Problems

The first-order optimality conditions of sparse bilinear least squares problems are studied. The so-called T-type and N-type stationary points for this problem are characterized in terms of tangent cone and normal cone in Bouligand and Clarke senses, and another stationarity concept called the coordinate-wise minima is introduced and discussed. Moreover, the L-like stationary point for this problem is introduced and analyzed through the newly introduced concept of like-projection, and the M-stationary point is also investigated via a complementarity-type reformulation of the problem. The relationship between these stationary points is discussed as well. It turns out that all stationary points discussed in this work satisfy the necessary optimality conditions for the sparse bilinear least squares problem.

math.OC↗

arXiv2025

Compressed Newton-direction-based Thresholding Methods for Sparse Optimization Problems

Thresholding algorithms for sparse optimization problems involve two key components: search directions and thresholding strategies. In this paper, we use the compressed Newton direction as a search direction, derived by confining the classical Newton step to a low-dimensional subspace and embedding it back into the full space with diagonal regularization. This approach significantly reduces the computational cost for finding the search direction while maintaining the efficiency of Newton-like methods. Based on this new search direction, we propose two major classes of algorithms by adopting hard or optimal thresholding: the compressed Newton-direction-based thresholding pursuit (CNHTP) and compressed Newton-direction-based optimal thresholding pursuit (CNOTP). We establish the global convergence of the proposed algorithms under the restricted isometry property. Experimental results demonstrate that the proposed algorithms perform comparably to several state-of-the-art methods in terms of success frequency and solution accuracy for solving the sparse optimization problem.

cs.IT↗

arXiv2025

Splitting Alternating Algorithms for Sparse Solutions of Linear Systems with Concatenated Orthogonal Matrices

A class of splitting alternating algorithms is proposed for finding the sparse solution of linear systems with concatenated orthogonal matrices. Depending on the number of matrices concatenated, the proposed algorithms are classified into the two-block splitting alternating algorithm (TSAA) and the multi-block splitting alternating algorithm (MSAA). These algorithms aim to decompose a large-scale linear system into two or more coupled subsystems, each significantly smaller than the original system, and then combine the solutions of these subsystems to produce the sparse solution of the original system. The proposed algorithms only involve matrix-vector products and reduced orthogonal projections. It turns out that the proposed algorithms are globally convergent to the sparse solution of a linear system if the matrix (along with the sparsity level of the solution) satisfies a coherence-type condition. Numerical experiments indicate that the proposed algorithms are very promising and can quickly and accurately locate the sparse solution of a linear system with significantly fewer iterations than several mainstream iterative methods.

cs.IT↗

arXiv2025

Reinforcement-learning-based Algorithms for Optimization Problems and Applications to Inverse Problems

We design a new iterative algorithm, called REINFORCE-OPT, for solving a general type of optimization problems. This algorithm parameterizes the solution search rule and iteratively updates the parameter using a reinforcement learning (RL) algorithm resembling REINFORCE. To gain a deeper understanding of the RL-based methods, we show that REINFORCE-OPT essentially solves a stochastic version of the given optimization problem, and that under standard assumptions, the searching rule parameter almost surely converges to a locally optimal value. Experiments show that REINFORCE-OPT outperforms other optimization methods such as gradient descent, the genetic algorithm, and particle swarm optimization, via its ability to escape from locally optimal solutions and its robustness to the choice of initial values. With rigorous derivations, we formally introduce the use of reinforcement learning to deal with inverse problems. By choosing specific probability models for the action-selection rule, we can also connect our approach to the conventional methods of Tikhonov regularization and iterative regularization. We take non-linear integral equations and parameter-identification problems in partial differential equations as examples to show how reinforcement learning can be applied in solving non-linear inverse problems. The numerical experiments highlight the strong performance of REINFORCE-OPT, as well as its ability to quantify uncertainty in error estimates and identify multiple solutions for ill-posed inverse problems that lack solution stability and uniqueness.

math.OC↗

arXiv2024

On NP-Hardness of $L_1/L_2$ Minimization and Bound Theory of Nonzero Entries in Solutions

The $L_1/L_2$ norm ratio has gained significant attention as a measure of sparsity due to three merits: sharper approximation to the $L_0$ norm compared to the $L_1$ norm, being parameter-free and scale-invariant, and exceptional performance with highly coherent matrices. These properties have led to its successful application across a wide range of fields. While several efficient algorithms have been proposed to compute stationary points for $L_1/L_2$ minimization problems, their computational complexity has remained open. In this paper, we prove that finding the global minimum of both constrained and unconstrained $L_1/L_2$ models is strongly NP-hard. In addition, we establish uniform upper bounds on the $L_2$ norm for any local minimizer of both constrained and unconstrained $L_1/L_2$ minimization models. We also derive upper and lower bounds on the magnitudes of the nonzero entries in any local minimizer of the unconstrained model, aiding in classifying nonzero entries. Finally, we extend our analysis to demonstrate that the constrained and unconstrained $L_p/L_q$ ($0 < p \leq 1, 1 < q < +\infty$) models are also strongly NP-hard.

math.OC↗

arXiv2024

Dynamic Thresholding Algorithm with Memory for Linear Inverse Problems

The relaxed optimal $k$-thresholding pursuit (ROTP) is a recent algorithm for linear inverse problems. This algorithm is based on the optimal $k$-thresholding technique which performs vector thresholding and error metric reduction simultaneously. Although ROTP can be used to solve small to medium-sized linear inverse problems, the computational cost of this algorithm is high when solving large-scale problems. By merging the optimal $k$-thresholding technique and iterative method with memory as well as optimization with sparse search directions, we propose the so-called dynamic thresholding algorithm with memory (DTAM), which iteratively and dynamically selects vector bases to construct the problem solution. At every step, the algorithm uses more than one or all iterates generated so far to construct a new search direction, and solves only the small-sized quadratic subproblems at every iteration. Thus the computational complexity of DTAM is remarkably lower than that of ROTP-type methods. It turns out that DTAM can locate the solution of linear inverse problems if the matrix involved satisfies the restricted isometry property. Experiments on synthetic data, audio signal reconstruction and image denoising demonstrate that the proposed algorithm performs comparably to several mainstream thresholding and greedy algorithms, and it works much faster than the ROTP-type algorithms especially when the sparsity level of signal is relatively low.

cs.IT↗

arXiv2023

Heavy-ball-based optimal thresholding algorithms for sparse linear inverse problems

Linear inverse problems arise in diverse engineering fields especially in signal and image reconstruction. The development of computational methods for linear inverse problems with sparsity is one of the recent trends in this field. The so-called optimal $k$-thresholding is a newly introduced method for sparse optimization and linear inverse problems. Compared to other sparsity-aware algorithms, the advantage of optimal $k$-thresholding method lies in that it performs thresholding and error metric reduction simultaneously and thus works stably and robustly for solving medium-sized linear inverse problems. However, the runtime of this method is generally high when the size of the problem is large. The purpose of this paper is to propose an acceleration strategy for this method. Specifically, we propose a heavy-ball-based optimal $k$-thresholding (HBOT) algorithm and its relaxed variants for sparse linear inverse problems. The convergence of these algorithms is shown under the restricted isometry property. In addition, the numerical performance of the heavy-ball-based relaxed optimal $k$-thresholding pursuit (HBROTP) has been evaluated, and simulations indicate that HBROTP admits robustness for signal and image reconstruction even in noisy environments.

math.NA↗

arXiv2023

Dynamic Orthogonal Matching Pursuit for Sparse Data Reconstruction

The orthogonal matching pursuit (OMP) is one of the mainstream algorithms for sparse data reconstruction or approximation. It acts as a driving force for the development of several other greedy methods for sparse data reconstruction, and it also plays a vital role in the development of compressed sensing theory for sparse signal and image reconstruction. In this paper, we propose the so-called dynamic orthogonal matching pursuit (DOMP) and enhanced dynamic orthogonal matching pursuit (EDOMP) algorithms which are more efficient than OMP for sparse data reconstruction from a numerical point of view. We carry out a rigorous analysis to establish the reconstruction error bound for DOMP under the restricted isometry property of the measurement matrix. The main result claims that the reconstruction error via DOMP can be controlled and measured in terms of the number of iterations, sparsity level of data, and the noise level of measurements. Moveover, the finite convergence of DOMP for a class of large-scale compressed sensing problems is also shown.

cs.IT↗

arXiv2022

Natural Thresholding Algorithms for Signal Recovery with Sparsity

The algorithms based on the technique of optimal $k$-thresholding (OT) were recently proposed for signal recovery, and they are very different from the traditional family of hard thresholding methods. However, the computational cost for OT-based algorithms remains high at the current stage of their development. This stimulates the development of the so-called natural thresholding (NT) algorithm and its variants in this paper. The family of NT algorithms is developed through the first-order approximation of the so-called regularized optimal $k$-thresholding model, and thus the computational cost for this family of algorithms is significantly lower than that of the OT-based algorithms. The guaranteed performance of NT-type algorithms for signal recovery from noisy measurements is shown under the restricted isometry property and concavity of the objective function of regularized optimal $k$-thresholding model. Empirical results indicate that the NT-type algorithms are robust and very comparable to several mainstream algorithms for sparse signal recovery.

cs.IT↗

arXiv2022

Heavy-Ball-Based Hard Thresholding Algorithms for Sparse Signal Recovery

The hard thresholding technique plays a vital role in the development of algorithms for sparse signal recovery. By merging this technique and heavy-ball acceleration method which is a multi-step extension of the traditional gradient descent method, we propose the so-called heavy-ball-based hard thresholding (HBHT) and heavy-ball-based hard thresholding pursuit (HBHTP) algorithms for signal recovery. It turns out that the HBHT and HBHTP can successfully recover a $k$-sparse signal if the restricted isometry constant of the measurement matrix satisfies $δ_{3k}<0.618 $ and $δ_{3k}<0.577,$ respectively. The guaranteed success of HBHT and HBHTP is also shown under the conditions $δ_{2k}<0.356$ and $δ_{2k}<0.377,$ respectively. Moreover, the finite convergence and stability of the two algorithms are also established in this paper. Simulations on random problem instances are performed to compare the performance of the proposed algorithms and several existing ones. Empirical results indicate that the HBHTP performs very comparably to a few existing algorithms and it takes less average time to achieve the signal recovery than these existing methods.

cs.IT↗

arXiv2021

Partial Gradient Optimal Thresholding Algorithms for a Class of Sparse Optimization Problems

The optimization problems with a sparsity constraint is a class of important global optimization problems. A typical type of thresholding algorithms for solving such a problem adopts the traditional full steepest descent direction or Newton-like direction as a search direction to generate an iterate on which a certain thresholding is performed. Traditional hard thresholding discards a large part of a vector when the vector is dense. Thus a large part of important information contained in a dense vector has been lost in such a thresholding process. Recent study [Zhao, SIAM J Optim, 30(1), pp. 31-55, 2020] shows that the hard thresholding should be applied to a compressible vector instead of a dense vector to avoid a big loss of information. On the other hand, the optimal $k$-thresholding as a novel thresholding technique may overcome the intrinsic drawback of hard thresholding, and performs thresholding and objective function minimization simultaneously. This motivates us to propose the so-called partial gradient optimal thresholding method in this paper, which is an integration of the partial gradient and the optimal $k$-thresholding technique. The solution error bound and convergence for the proposed algorithms have been established in this paper under suitable conditions. Application of our results to the sparse optimization problems arising from signal recovery is also discussed. Experiment results from synthetic data indicate that the proposed algorithm called PGROTP is efficient and comparable to several existing algorithms.

math.OC↗

arXiv2021

Newton-Type Optimal Thresholding Algorithms for Sparse Optimization Problems

Sparse signals can be possibly reconstructed by an algorithm which merges a traditional nonlinear optimization method and a certain thresholding technique. Different from existing thresholding methods, a novel thresholding technique referred to as the optimal $k$-thresholding was recently proposed by Zhao [SIAM J Optim, 30(1), pp. 31-55, 2020]. This technique simultaneously performs the minimization of an error metric for the problem and thresholding of the iterates generated by the classic gradient method. In this paper, we propose the so-called Newton-type optimal $k$-thresholding (NTOT) algorithm which is motivated by the appreciable performance of both Newton-type methods and the optimal $k$-thresholding technique for signal recovery. The guaranteed performance (including convergence) of the proposed algorithms are shown in terms of suitable choices of the algorithmic parameters and the restricted isometry property (RIP) of the sensing matrix which has been widely used in the analysis of compressive sensing algorithms. The simulation results based on synthetic signals indicate that the proposed algorithms are stable and efficient for signal recovery.

eess.SP↗

arXiv2021

Dual-density-based reweighted $\ell_{1}$-algorithms for a class of $\ell_{0}$-minimization problems

The optimization problem with sparsity arises in many areas of science and engineering such as compressed sensing, image processing, statistical learning and data sparse approximation. In this paper, we study the dual-density-based reweighted $\ell_{1}$-algorithms for a class of $\ell_{0}$-minimization models which can be used to model a wide range of practical problems. This class of algorithms is based on certain convex relaxations of the reformulation of the underlying $\ell_{0}$-minimization model. Such a reformulation is a special bilevel optimization problem which, in theory, is equivalent to the underlying $\ell_{0}$-minimization problem under the assumption of strict complementarity. Some basic properties of these algorithms are discussed, and numerical experiments have been carried out to demonstrate the efficiency of the proposed algorithms. Comparison of numerical performances of the proposed methods and the classic reweighted $\ell_1$-algorithms has also been made in this paper.

math.OC↗

arXiv2020

Analysis of Optimal Thresholding Algorithms for Compressed Sensing

The optimal $k$-thresholding (OT) and optimal $k$-thresholding pursuit (OTP) are newly introduced frameworks of thresholding techniques for compressed sensing and signal approximation. Such frameworks motivate the practical and efficient algorithms called relaxed optimal $k$-thresholding ($\textrm{ROT}ω$) and relaxed optimal $k$-thresholding pursuit ($\textrm{ROTP}ω$) which are developed through the tightest convex relaxations of OT and OTP, where $ω$ is a prescribed integer number. The preliminary numerical results demonstrated in \cite{Z19} indicate that these approaches can stably reconstruct signals with a wide range of sparsity levels. However, the guaranteed performance of these algorithms with parameter $ ω\geq 2 $ has not yet established in \cite{Z19}. The purpose of this paper is to show the guaranteed performance of OT and OTP in terms of the restricted isometry property (RIP) of nearly optimal order for the sensing matrix governing the $k$-sparse signal recovery, and to establish the first guaranteed performance result for $\textrm{ROT}ω$ and $\textrm{ROTP}ω$ with $ ω\geq 2.$ In the meantime, we provide a numerical comparison between ROTP$ω$ and several existing thresholding methods.

math.OC↗

arXiv2020

Improved RIP-Based Bounds for Guaranteed Performance of two Compressed Sensing Algorithms

Iterative hard thresholding (IHT) and compressive sampling matching pursuit (CoSaMP) are two types of mainstream compressed sensing algorithms using hard thresholding operators for signal recovery and approximation. The guaranteed performance for signal recovery via these algorithms has mainly been analyzed under the condition that the restricted isometry constant of a sensing matrix, denoted by $ δ_K$ (where $K$ is an integer number), is smaller than a certain threshold value in the interval $(0,1).$ The condition $ δ_{K}< δ^*$ for some constant $ δ^* \leq 1 $ ensuring the success of signal recovery with a specific algorithm is called the restricted-isometry-property-based (RIP-based) bound for guaranteed performance of the algorithm. At the moment, the best known RIP-based bound for the guaranteed recovery of $k$-sparse signals via IHT is $δ_{3k}< 1/\sqrt{3}\approx 0.5774,$ and the bound for guaranteed recovery via CoSaMP is $δ_{4k} < 0.4782. $ A fundamental question in this area is whether such theoretical results can be further improved. The purpose of this paper is to affirmatively answer this question and rigorously show that the RIP-based bounds for guaranteed performance of IHT can be significantly improved to $ δ_{3k} < (\sqrt{5}-1)/2 \approx 0.618, $ and the bound for CoSaMP can be improved and pushed to $ δ_{4k}< 0.5102. $ These improvements are achieved through a deep property of the hard thresholding operator.

eess.SP↗

arXiv2020

Newton-Step-Based Hard Thresholding Algorithms for Sparse Signal Recovery

Sparse signal recovery or compressed sensing can be formulated as certain sparse optimization problems. The classic optimization theory indicates that the Newton-like method often has a numerical advantage over the gradient method for nonlinear optimization problems. In this paper, we propose the so-called Newton-step-based iterative hard thresholding (NSIHT) and the Newton-step-based hard thresholding pursuit (NSHTP) algorithms for sparse signal recovery and signal approximation. Different from the traditional iterative hard thresholding (IHT) and hard thresholding pursuit (HTP), the proposed algorithms adopts the Newton-like search direction instead of the steepest descent direction. A theoretical analysis for the proposed algorithms is carried out, and some sufficient conditions for the guaranteed success of sparse signal recovery via these algorithms are established. Our results are shown under the restricted isometry property which is one of the standard assumptions widely used in the field of compressed sensing and signal approximation. The empirical results obtained from synthetic data recovery indicate that the proposed algorithms are efficient signal recovery methods. The numerical stability of our algorithms in terms of the residual reduction is also investigated through simulations.

math.OC↗

arXiv2019

Optimal $k$-thresholding algorithms for sparse optimization problems

The simulations indicate that the existing hard thresholding technique independent of the residual function may cause a dramatic increase or numerical oscillation of the residual. This inherit drawback of the hard thresholding renders the traditional thresholding algorithms unstable and thus generally inefficient for solving practical sparse optimization problems. How to overcome this weakness and develop a truly efficient thresholding method is a fundamental question in this field. The aim of this paper is to address this question by proposing a new thresholding technique based on the notion of optimal $k$-thresholding. The central idea for this new development is to connect the $k$-thresholding directly to the residual reduction during the course of algorithms. This leads to a natural design principle for the efficient thresholding methods. Under the restricted isometry property (RIP), we prove that the optimal thresholding based algorithms are globally convergent to the solution of sparse optimization problems. The numerical experiments demonstrate that when solving sparse optimization problems, the traditional hard thresholding methods have been significantly transcended by the proposed algorithms which can even outperform the classic $\ell_1$-minimization method in many situations.

cs.IT↗